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Key findings 




This study examined quality improvement activities among Iowa’s early childhood education 
programs and the relationships between those activities and ratings on the state’s Quality 
Rating System (QRS). The study found that: 

Early childhood education programs participating in Iowa’s QRS used in-person training 
more than other types of quality improvement activities, and the most common topic of 
professional development was health and safety practices. 

Financial supports were the most common quality improvement support received by 
programs and staff. 

Participation in quality improvement activities was often hindered by logistical barriers 
such as travel distance, training schedule, and cost. 

Having staff complete at least 15 hours of training per year and professional development 
on management topics were both positively related to increases in Iowa QRS rating. 
Professional development on child development topics was negatively related to rating 


increases. 
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Summary 


States have prioritized documenting and improving the quality of early childhood edu- 
cation programs by developing quality rating and improvement systems (QRISs). 1 QRISs 
have two major components: systematic ratings of program quality, and integrated quality 
improvement supports and activities. This report describes the quality improvement efforts 
of early childhood education programs participating in Iowa’s voluntary quality rating and 
improvement system, the Iowa Quality Rating System (QRS). 

To effectively allocate resources to support quality improvement efforts, state admin- 
istrators need information about how programs use quality improvement supports and 
activities. The Iowa Quality Rating System Oversight Committee, which includes repre- 
sentatives of early childhood education programs and several state agencies and organiza- 
tions in Iowa, requested Regional Educational Laboratory (REL) Midwest’s assistance in 
examining program quality improvement efforts in Iowa and how these efforts relate to 
increases in Iowa QRS ratings. 

Data for the study came from two sources: administrative records on programs participat- 
ing in Iowa’s QRS ratings system and responses from administrators in 390 of 600 early 
childhood education programs invited to respond to a 2014 survey about quality improve- 
ment supports and activities in the previous year (a 65 percent response rate). 

Survey responses indicated that staff in almost all programs participating in the Iowa QRS 
(97 percent) received training the previous year. At least some staff in almost two-thirds 
of programs (65 percent) received coaching the previous year. The most common topic 
of professional development reported by survey respondents was health and safety prac- 
tices, followed by child development and classroom practices. Staff in approximately one- 
third of programs (36 percent) pursued a degree or credential the previous year, and about 
three-quarters of programs offered orientation (78 percent) and mentoring (77 percent) to 
new staff. The amount and types of quality improvement activities varied by staff type; for 
example, assistants, especially those working in home-based settings, participated in few 
activities. 

Among the supports received by programs to participate in quality improvement activities, 
survey respondents reported more use of financial incentives (for example, financial sup- 
ports for education or training costs, or release time) than of nonfinancial incentives (for 
example free or discounted materials). Programs also reported that logistical factors such as 
distance and travel time to training sites, inconvenient training schedules, training costs, 
and the availability of training on specific topics were the most common barriers to partic- 
ipation in quality improvement activities. 

In addition, ratings tended to increase over time among programs that participated in the 
Iowa QRS long enough to receive a second rating. Ninety-seven percent of programs that 
had two ratings either maintained the same rating (52 percent) or received a higher rating 
(45 percent). Of the 10 quality improvement activities, 3 were significantly related to rating 
increases. Ratings tended to increase over time for programs that provided key staff with 
15 or more training hours per year, but the increases may be due partly to the fact that 
the training hours themselves are factored into a program’s rating. Ratings also tended 
to increase over time for programs in which providers received professional development 


on topics related to program management. In contrast, staff professional development on 
child development topics was negatively related to rating increases. 

The findings of this study provide an overview of the landscape of quality improvement 
activities among early childhood education programs participating in the Iowa QRS and 
their relationship with increased ratings. The results suggest that in-person training is the 
most common type of quality improvement activity in Iowa, perhaps partly because annual 
in-person training hours are required to maintain home provider registration or center 
licensing. Yet fewer than half the programs met the study team’s adaptation of the Nation- 
al Institute for Early Education Research benchmark of providing an average of at least 
15 hours of training for key staff (Barnett, Carolan, Squires, Brown, & Horowitz, 2014). 
Meeting this benchmark was positively and statistically significantly related to rating 
increases, suggesting that the benchmark is meaningful for improving program quality 
within the Iowa QRS. However, many survey respondents identified long distance and 
travel time as barriers to participation in quality improvement activities, and web-based 
trainings may be one solution to overcoming these barriers. In addition, about two-thirds 
of programs reported that staff received coaching, though only 15 percent of programs 
reported that staff received ongoing coaching, and coaching was not statistically signifi- 
cantly related to increased ratings. 

Among topics in training and coaching, only professional development for providers on 
management topics was related to rating increases, although the most common were health 
and safety and child development. This positive relationship suggests that the state may 
wish to conduct further research to examine whether the management training is leading 
to improvements in program quality or whether other factors may explain the observed 
relationships. Also, the counterintuitive negative relationship between teacher profession- 
al development in child development topics and ratings should be interpreted with caution 
because the results may be explained by other factors. For example, the programs that did 
not focus on child development topics may have staff with prior expertise in child develop- 
ment, and that expertise might also help drive rating increases. 

These findings can help Iowa QRS administrators in planning and allocating resources to 
improve program quality. The findings can also help administrators of quality rating and 
improvement systems in other states understand more clearly the types of quality improve- 
ment activities to which programs are drawn naturally, as well as factors that may facilitate 
or impede programs in their pursuit of quality. 
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Why this study? 


States have prioritized documenting and improving the quality of early childhood educa- 
tion programs by developing quality rating and improvement systems (QRISs). This report 
describes the quality improvement efforts of early childhood education programs partic- 
ipating in Iowa’s Quality Rating System (QRS; see box 1 for definitions of key terms). It 
identifies supports and barriers to quality improvement and examines how quality improve- 
ment supports and activities relate to changes in program quality ratings across time. 


Box 1. Key terms 

Center. A center-based early childhood education program offering education or care in a group 
setting that is not located in a home, including licensed child care centers, preschools, public 
preK, and center-based Head Start and Early Head Start programs. 

Domain. Early childhood education programs that participate in the Iowa Quality Rating System 
earn points by meeting criteria in specific domains: professional development, health and 
safety, environment, and family and community partnerships for centers and homes, and also 
leadership and administration for centers only. 

Home. A home-based early childhood education program offering education or care in a group 
setting located in a registered (licensed) provider’s home, including small and large family child 
care homes. 

Iowa Quality Rating System (Iowa QRS). Iowa’s quality rating and improvement system. Pro- 
grams earn points by meeting criteria in specific domains. The sum of the domain scores 
determines the program’s rating; an independent observation is also required for a level 5 
rating. Program participation is voluntary. 

Levels of barriers to quality improvement. Information derived from survey items that asked 
respondents to rate the effect that nine factors have had on their program’s level of participa- 
tion in professional development and other quality improvement activities, with ratings of no 
effect (1), small effect (2), medium effect (3), or large effect (4). 

Odds ratio , The likelihood of an outcome occurring (in this case, an increase in the QRS rating) 
through exposure to some event (professional development on management topics) in compar- 
ison with the likelihood of the outcome occurring without exposure to the event. An odds ratio 
of 1 typically indicates no relationship, whereas odds ratios greater than 1 indicate positive 
relationships, and odds ratios less than 1 indicate negative relationships. (See appendix C for 
details on data and methods.) 

Program. An early childhood education program (center or home) offering education or care in 
a group setting. 

Quality rating and improvement system (QRIS) A system that uses multiple data sources on 
early childhood education programs to calculate ratings of program quality to document and 
disseminate reliable information about program quality and encourage quality improvement. 

Staff and key staff. Staff refers to managers, teachers, and assistant teachers in centers and 
providers and assistants in homes. Key staff refers to managers and teachers in centers and 
providers in homes. (See box Cl in appendix C for additional detail.) 
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What is known about quality rating and improvement systems 


Research highlights the importance of high-quality early childhood education to reduce 
school readiness gaps and improve developmental outcomes among children (for example, 
Howes et al., 2008; Yoshikawa & Zaslow, 2013). Accordingly, national policymakers have 
made early childhood education program quality a priority (Yoshikawa & Zaslow, 2013), 
and states have developed QRISs. (See appendix A for a more detailed review of the liter- 
ature on QRISs.) The number of states with a QRIS has grown rapidly in the last decade, 
and by 2014 all states but one were planning, piloting, or implementing a QRIS (Build 
Initiative & Child Trends, 2015; QRIS National Learning Network, 2015). All seven states 
in the Regional Educational Laboratory (REL) Midwest Region (Illinois, Indiana, Iowa, 
Michigan, Minnesota, Ohio, and Wisconsin) have established a QRIS, although the states 
are at different stages of implementation. 

QRISs have two primary components: systematic ratings of program quality, and integrat- 
ed quality improvement supports and activities. The second component aims to help pro- 
grams make improvements and raise quality and QRIS ratings (Schaack, Tarrant, Boiler, 
& Tout, 2012). States differ in the type and amount of quality improvement supports 
and activities available to programs and in the extent to which the supports and activ- 
ities are required or accessible to all participating programs (Holod, Faria, Weinberg, & 
Howard, 2015). Quality improvement activities include professional development (train- 
ing and coaching), continuing education, and new staff orientation. Quality improve- 
ment supports include both financial and nonfinancial incentives to encourage programs 
and their staff to improve program quality (see box 2 for detailed descriptions of quality 
improvement supports and activities and information about what Iowa offers). Profession- 
al development topics describe the content of training and coaching. (See appendix C 
for additional information about the professional development topics examined in this 
study.) States also differ in their approaches to systematic ratings, including differences in 
the types of data used for the ratings, the methods used to calculate ratings, the program 
types included in the QRIS, and the way ratings are disseminated (Build Initiative & 
Child Trends, 2015). 


States differ in 
the type and 
amount of quality 
improvement 
supports and 
activities available 
to programs 


State QRIS administrators need information about how programs use quality improvement 
supports and activities to improve services. To date, much of the research on QRISs has 
focused on validating the QRIS ratings. These validation studies examine whether ratings 
from a QRIS accurately measure program quality as intended and differentiate higher and 
lower quality programs (for example, Hawkinson et al., 2015; Hestenes et al., 2015; Karoly, 
Zellman, & Perlman, 2013; Le, Schaack, & Setodji, 2015; Sabol & Pianta, 2015; Tout et al., 
2011). Most validation studies have found that programs with higher ratings demonstrate 
better performance on observational measures of the quality of classroom interactions, but 
QRISs have yet to differentiate programs in ways that consistently predict child outcomes 
(Karoly, 2014). 

States rely on the quality improvement supports and activities component of the QRIS to 
help programs attain higher QRIS ratings. Although many research studies have examined 
the relationship between quality improvement supports and activities and classroom out- 
comes, few studies have examined the relationship between specific quality improvement 
supports and activities and QRIS ratings (see appendix A). Using administrative data 
on supports and activities offered in Miami and QRIS ratings, Yazejian and Iruka (2015) 
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Box 2. Description of quality improvement activities and supports 


Quality improvement activities 

Quality improvement activities are activities intended to improve the quality of program services, such as profession- 
al development (training and coaching), continuing education, and new staff orientation. 

• Professional development. A blanket term for training and coaching. This study examines professional develop- 
ment topics covered in training and coaching. Training covers professional development instruction (or work- 
shops) provided to a group of early childhood education program staff on specific job-related topics, conducted 
in person or in a web-based format. In Iowa, regional Child Care Resource and Referral agencies and other 
entities offer training for any early childhood education program . 1 Programs may also receive training from 
other organizations. Coaching covers individualized professional development support (often called technical 
assistance or consultation) provided by an individual to early childhood education program staff, conducted in 
person, using video or web-based platforms, or through informal communications such as telephone, email, and 
text messaging. Coaching may include classroom support for teaching staff and program support for manage- 
ment staff. Ongoing coaching refers to coaching occurring at least monthly. 

Example from Iowa: Early childhood education programs in Iowa can request free consultations on the QRS 
rating process and quality improvement plans (with a QRS consultant), and on health and safety practices (with 
a nurse consultant). Programs may receive coaching from agencies not affiliated with the state. 

• Continuing education. Credit-bearing coursework toward any credential, degree, or certificate by actively employed 
early childhood education program staff. 

Example from Iowa: Staff in Iowa programs can obtain early childhood apprenticeship certificates, Child 
Development Associate credentials, administrator credentials that do not yield a college degree, and adminis- 
trator credentials for program directors. Iowa’s higher education institutions also offer associate’s, bachelor’s, 
and master’s degrees in early childhood education. 

• New staff orientation. The provision of information, training, and other supports to new staff. Includes mentoring 
and guidance and support offered to new staff by colleagues or a professional mentor. 

Example from Iowa: Iowa State University Extension offers free orientation materials to Iowa’s center-based 
programs. Programs may also provide their own orientation or access other orientation resources. 

Quality improvement supports 

Quality improvement supports are resources intended to support quality improvement efforts, including financial 
incentives and nonfinancial incentives received by programs or by individual staff. 

• Financial incentives. Monetary incentives that are available both to programs (such as grants, awards, and tiered 
reimbursements) and to individual staff members (such as scholarships). 

Example from Iowa: Early childhood education programs in Iowa can receive monetary awards for completing 
the Iowa QRS rating process (based on their rating level). Program staff can apply for Teacher Education and 
Compensation Helps Early Childhood Iowa scholarships for continuing education, and programs can apply for 
grants or other support from various entities . 2 

• Nonfinancial incentives. Nonmonetary resources to support quality improvement efforts, including materials 
(such as books or games) and equipment (such as playground structures or cribs) that programs may receive for 
free or at a discount. 

Example from Iowa: Programs in Iowa may be eligible for free or discounted materials or equipment from the state. 

Notes 

1. Iowa State University and Early Childhood Iowa also offer training in Iowa. Information on Iowa-specific supports and activities was 
provided by members of the Iowa Quality Rating System (Iowa QRS) Oversight Steering Committee. 

2 . Teacher Education and Compensation Helps (T.E.A.C.H.) is a national initiative to increase access to higher education among early 
educators working with young children. It provides scholarships and other supports for degree and certification programs. Each state 
operates its own T.E.A.C.H. program. More information about T.E.A.C.H. in Iowa is available at: http://www.iowaaeyc.org/teach.cfm. 
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found that scholarships for continuing education were associated with rating increases 
for center-based early childhood education programs but not for home-based programs. 
A study in Pennsylvania found higher QRIS rating increases among low-rated programs 
that participated in an intensive quality improvement activity program involving coach- 
ing, improvement funds, and other supports (Warner-Richter, Lowe, Tout, Epstein, & Li, 
2016). Additional research is needed on a broader range of supports and activities and 
representing other geographic areas. States need information about each type of quality 
improvement support and activity among programs in QRIS and about the relationship 
between these supports and activities and QRIS rating increases, so that administrators 
can determine how to allocate resources most effectively to support quality improvement 
efforts. 


Iowa’s Quality Rating System 

Members of the Iowa Quality Rating System Oversight Committee 2 requested REL Mid- 
west’s assistance in examining quality improvement efforts of early childhood educa- 
tion programs and the relationship of these efforts to increases in programs’ Iowa QRS 
ratings. The Iowa QRS was launched in 2006 and recalibrated in 2010 (see box 3 and 
appendix B for details about the Iowa QRS ratings). The Iowa QRS is administered 
by the Iowa Department of Human Services and governed by the Iowa Quality Rating 
System Oversight Committee, which includes representatives of early childhood educa- 
tion programs and several state agencies. Members of the Iowa Quality Rating System 
Oversight Committee identified key quality improvement supports and activities offered 
in Iowa (see box 2) and then sought REL Midwest’s assistance in examining how the 
programs used these supports and activities and how that use relates to changes in Iowa 
QRS ratings. 


State-level 
stakeholders can 
use the study 
findings to inform 
decisions about 
the funding and 
distribution 
of quality 
improvement 
supports and 
activities 


The primary audience for this report is Iowa QRS stakeholders, including members of the 
Iowa Quality Rating System Oversight Committee, state agency administrators, policy- 
makers, regional Child Care Resource and Referral agency staff, technical assistance pro- 
viders, and higher education institutions that offer early childhood education degree and 
certification programs. State-level stakeholders can use the study findings to inform deci- 
sions about the funding and distribution of quality improvement supports and activities. 
Early childhood education program managers in Iowa may be interested in reviewing the 
findings of this report as they consider their decisions about quality improvement activi- 
ties. QRIS stakeholders in other states may also be interested in the results of this study 
and in collecting similar data on quality improvement activities among programs in their 
own QRISs. The diverse community of QRIS advocates and researchers is another audi- 
ence for this study, particularly given the limited research to date on the issues explored in 
this report. 
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Box 3. Iowa Quality Rating System ratings 


Early childhood education programs eligible to participate in the Iowa Quality Rating System 
(QRS) fall into two categories: 

• Centers that offer any type of center-based early childhood development care, including 
licensed child care centers, preschools, public preK, center-based Head Start or Early 
Head Start, and after-school programs. Centers typically rely on teachers and assistant 
teachers in each classroom and a manager who may also be a teacher. 

• Homes that are registered family child care homes. Homes rely on a provider who teaches 
and manages the program, sometimes with the help of assistants. 

Participation in the Iowa QRS is voluntary, and ratings are valid for two years. Iowa counts 
1,438 licensed centers, 541 public schools with preK programs, and 3,227 registered child 
care homes operating within the state (Iowa Department of Human Services, 2015; U.S. 
Department of Education, National Center for Education Statistics, 2015). In 2014, 1,294 
programs were participating in the Iowa QRS, including 646 centers (33 percent of licensed 
centers and preK programs) and 648 homes (20 percent of registered homes). 

The Iowa QRS has five ratings, ranging from 1 (lowest) to 5 (highest). The criteria for each 
rating differ for centers and homes. A hybrid approach is used for calculating ratings based on 
a combination of mandatory standards that all programs must meet for ratings 1 and 2 and 
more flexible point-based requirements for the higher ratings. Programs earn points by meeting 
criteria in specific domains (professional development, health and safety, environment, and 
family and community partnerships for centers and homes, and also leadership and admin- 
istration for centers only). The sum of the domain scores determines the program’s rating, 
and an independent observation is also required for a rating of 5. (See appendix B for rating 
standards and criteria.) 


What the study examined 


The study team developed the following research questions for this study in partnership 
with stakeholders in Iowa. Among early childhood education programs participating in 
the Iowa QRS: 

1. What are the quality improvement activities in which they participate? 

2. What topics are covered in professional development? 

3. What are the supports and barriers to participation in quality improvement activities? 

4. What information do they use to make decisions about quality improvement activities? 

5. Do ratings rise among programs that renew their Iowa QRS participation? 

6. What is the relationship between quality improvement supports and activities and 
increases in QRS ratings and domain scores? 

The research questions reflect the Iowa Quality Rating System Oversight Committee’s 
need to learn how programs participating in the Iowa QRS approach quality improve- 
ments, to identify supports and barriers to quality improvement efforts, and to determine if 
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there are correlations between quality improvement approaches and outcomes. To address 
these questions, the study team examined programs’ responses to a survey administered in 
October and November 2014 and the statistical relationships between these responses and 
programs’ QRS ratings across time (see box 4 for details; more details on methodology are 
in appendix C). 


Box 4. Data and methods 

Data 

The study team addressed the research questions by analyzing two types of data. 

• Programs’ responses to a survey about quality improvement supports and activities during 
the previous year. In fall 2014 Iowa’s Department of Human Services administered a survey 
to a stratified random sample of 600 early childhood education programs throughout the 
state, and 390 of 600 randomly sampled programs completed the survey, a 65 percent 
response rate. (Survey items are provided in appendix D; two programs did not answer 
all items, resulting in analyses with 388 programs.) The study team worked with the Iowa 
Quality Rating System Oversight Committee to develop the survey instrument and pro- 
vided technical assistance during data collection by supplying guidance on sampling and 
data collection procedures. The sample of respondents mirrored the population of QRS 
programs in terms of ratings, program type (center or home), and region in Iowa, but the 
sample underrepresented school-based centers and overrepresented community-based 
centers. The survey collected data about quality improvement activities among staff (man- 
agers, teachers, and assistant teachers in centers and about providers and assistants in 
homes). The number of managers, teachers, and assistant teachers varies in centers, and 
homes include those with and without assistants. (See appendix C for definitions of staff 
types and details about the data.) 

• Administrative data on program characteristics and the iowa QRS ratings. These data are 
collected by Iowa’s Department of Human Services. Data include program type, location, 
region in the state, funding sources, Iowa QRS ratings as of September 2013 and Decem- 
ber 2014, and domain scores used for the ratings. 

Methods 

Research questions 1-4 ask about the quality improvement activities of Iowa’s early childhood 
education programs, the supports and barriers that programs experience in improving quality, 
and the sources of the information programs use to make decisions about quality improvement 
activities. These questions were addressed using descriptive statistics and creating tables and 
charts summarizing the survey responses. The study team also summarized survey respons- 
es according to program characteristics. To address research question 5, which asks about 
rating increases among re-rated programs, the study team used descriptive cross-tabulations 
of ratings at two points in time (September 2013 and December 2014). Logistic regression 
analysis was used to address research question 6 on whether specific activities and supports 
are related to increases in Iowa QRS ratings. 
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What the study found 


The following sections summarize the study findings, drawing from analyses of survey and 
administrative data. (More detailed findings are provided in appendix E.) 


Among participating programs, the most common quality improvement activity reported by survey 
respondents was in-person training 


Staff in almost all programs (97 percent) had participated in some type of training 
during the year before they completed the survey. Nearly all programs support staff par- 
ticipation in training (97 percent), but slightly less than half of programs (47 percent) 
reported that key staff met or exceeded the quality benchmark of 15 average annual hours 
of training (figure 1; see also table El in appendix E). 3 Most staff in the surveyed programs 
(95 percent) participated in in-person training, and nearly half of staff (49 percent) in the 
surveyed programs participated in web-based training. Fewer programs (21 percent) report- 
ed providing follow-up supports after either in-person or web-based training. 

Survey responses were examined separately for centers and homes and by Iowa QRS rating 
(see appendix E for information on difference in training by initial QRS rating). Homes 
and centers did not differ significantly in the attendance of staff at in-person training or 
any training at all, but a higher percentage of staff at centers (60 percent) than at homes 
(39 percent) participated in web-based training (see table El in appendix E). A higher 
percentage of staff in centers than of staff in homes also received follow-up after trainings 
(see table El in appendix E). Although participation in quality improvement activities was 
generally higher among centers than among homes, a larger percentage of key staff in 
homes met the 15-hour benchmark. This may be partly because the state requires more 


Most staff in the 
surveyed programs 
(95 percent) 
participated in 
in-person training , 
and nearly half of 
staff (49 percent) 
in the surveyed 
programs 
participated in 
web-based training 


Figure 1. Almost all programs participating in the Iowa Quality Rating System 
provided training to staff, but just slightly less than half provided an average of at 
least 15 hours of training to key staff, fall 2014 


Did not provide training 
3% 


A 




m 

Provided training to 
at least some staff 
97% 

J 



f 


n = 382 


Average training 
of 15 hours or 
more for key staff 
47% 



Note: The National Institute for Early Education Research identifies a minimum of 15 hours of training annually 
for teachers as a quality benchmark (Barnett et al., 2014). This study adapted the benchmark to an average of 
15 hours of training during the previous year for key staff. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 
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hours of training annually for registered home providers than for staff in licensed centers. 4 
Furthermore, most homes have only a single staff member (the provider), whereas centers 
typically have multiple staff and may distribute training opportunities across more staff. 

At least some staff in 65 percent of programs in the Iowa QRS received coaching. The 
degree to which program staff worked with professional coaches varied. Few programs 
(15 percent) had ongoing coaching, defined as coaching that occurs once a month or more 
(figure 2; see also table El in appendix E). Among all surveyed programs, 57 percent had 
staff who had worked with coaches in person, and 34 percent had staff who had worked with 
coaches using informal communication methods, such as telephone, email, or text messag- 
es. Only 7 percent of programs used web-based or video coaching for quality improvement. 


Although centers and homes did not differ in whether staff received at least some coaching 
in the previous year, a higher share of staff at centers (19 percent) than of staff at homes 
(11 percent) received ongoing coaching once a month or more (see table El in appendix 
E). Programs’ initial QRS rating was not significantly associated with the provision of any 
coaching during the previous year, but the percentage of programs in which staff received 
coaching every month or more frequently was highest among programs with a rating of 5. 

Staff at approximately one-third of programs were pursuing a degree or a credential. At 
least some staff at 36 percent of the programs participated in continuing education (see 
table E2 in appendix E). The share of staff pursuing a degree or credential was much higher 
among centers (55 percent) than among homes (19 percent). Participation in continuing 
education was more common among staff in programs with an initial QRS rating of 5 
than among staff in programs with lower ratings. 


Among all surveyed 
programs, 

57 percent had 
staff who had 
worked with 
coaches in person, 
and 34 percent 
had staff who 
had worked 
with coaches 
using informal 
communication 
methods, such as 
telephone, email, 
or text messages 


About three-quarters of programs offered orientation and mentoring to new staff. 
Among programs, 78 percent provided orientation to new staff (excluding homes with no 
assistants), and 77 percent offered mentoring (see table E2 in appendix E). A large majority 
of centers provided orientation (93 percent) and mentoring (92 percent) to new staff, but 


Figure 2. Two-thirds of programs participating in the Iowa Quality Rating System 
provided coaching to staff, but few programs had ongoing coaching once a month 
or more, fall 2014 



n = 385 



n = 377 


Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 
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the shares were less than a third among homes with assistants (33 percent and 30 percent). 
The provision of orientation and informal mentoring to new staff rose with higher Iowa 
QRS ratings. The duration of orientation varied, generally ranging from one to four hours. 


The amount and type of quality improvement activities varied by staff type, with 
assistants, especially in homes, participating in few activities. Assistants, especially in 
homes, tended to receive less training and coaching than other types of staff (figure 3). 
More than half of homes with assistants provided no training to the assistants, and assis- 
tants in homes who received training averaged fewer hours than any other type of staff. 

Only 14 percent of homes with assistants provided coaching to the assistants, while 
39 percent of centers provided coaching to at least some assistants. The percentage of staff 
in centers who engaged in continuing education was higher among teachers (38 percent) 
and assistant teachers (43 percent) than among managers (19 percent), whereas in homes 
the percentage was higher among providers (19 percent) than among assistants (5 percent). 

The most common topic of professional development reported by survey respondents was health 
and safety practices 


More than half 
of homes with 
assistants provided 
no training to 
the assistants, 
and assistants 
in homes who 
received training 
averaged fewer 
hours than any 
other type of staff 


Almost all programs (92 percent ) reported that their staff received professional develop- 
ment (either training or coaching) on health and safety practices in the year before they 
completed the survey. Other common topics of professional development among program 
staff were child development (85 percent of programs) and classroom practices (76 percent; 
figure 4; see also table E3 in appendix E). 


Figure 3. Among programs participating in the Iowa Quality Rating System, 
providers in homes received more training on average than did any type of staff in 
centers and far more training than assistants in homes, fall 2014 


Average training time (hours) 
25 “I 

20 - 


15 - 

10 - 


5 - 

0 

Providers Assistants 

(n = 179) (n = 49) 

Homes Centers 

Note: The number of programs that responded in each staff category varies because some programs do not 
have all listed staff types. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 
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Figure 4. Among survey respondents in programs participating in the Iowa Quality 
Rating System, health and safety practices was the most common topic of 
professional development among staff, followed by child development, fall 2014 


Percent of programs with staff who received professional development on each topic 



Health and Child Classroom Quality Program Program Family 

safety practices development practices Rating System self-assessment management engagement 

participation 


Note: n = 376. See table C6 in appendix C for definitions of the professional development (coaching and 
training) topics. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Fluman 
Services, in partnership with regional Child Care Resource and Referral agencies. 


In five of the seven topics investigated in the survey, the percentage of staff receiving pro- 
fessional development was higher in centers than in homes, with the differences partic- 
ularly large for classroom practices, program self-assessment, and program management 
topics (see table E3 in appendix E). The percentage of programs in which staff received 
professional development differed considerably by Iowa QRS rating for classroom practices, 
program self-assessment, and family engagement. The percentage was particularly high for 
programs with an initial rating of 5. 

Managers and providers in homes received professional development in a broad range of 
classroom- and program-related topics; assistants rarely received professional development 
on topics other than health and safety practices (figure 5; see also table E4 in appendix E). 
In centers, teachers and assistant teachers most commonly received professional develop- 
ment on classroom-related topics. 

Survey respondents indicated that financial supports were the most common type of quality 
improvement support received by programs and staff but that travel distance, scheduling, cost, and 
other logistical barriers often impeded participation 


In five of the seven 
topics investigated 
in the survey, 
the percentage 
of staff receiving 
professional 
development 
was higher in 
centers than 
in homes, with 
the differences 
particularly large 
for classroom 
practices, program 
self-assessment, 
and program 
management topics 


Among quality improvement supports received by programs, financial incentives were 
more common than nonfinancial incentives. Approximately three-quarters of programs 
reported receiving at least one type of incentive in the year before the survey (74 percent; see 
table E5 in appendix E). More than half the programs responding to the survey (56 percent) 
reported receiving monetary awards for achieving or maintaining their Iowa QRS ratings. 5 
Almost two-fifths of programs (39 percent) received a grant to support specific quality 
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Figure 5. Among programs participating in the Iowa Quality Rating System, few 
assistants in homes received professional development on topics other than health 
and safety practices, fall 2014 


Percent of homes with assistants in which assistants received professional development on each topic 
100 


75 - 


50 - 



Health and Child Classroom Family Program Quality Program 

safety practices development practices engagement management Rating System self-assessment 

participation 


Managers and 
providers in 
homes received 
professional 
development in 
a broad range of 
classroom- and 
program-related 
topics; assistants 
rarely received 
professional 
development 
on topics other 
than health and 
safety practices 


Note: n = 58. See table C6 in appendix C for definitions of the professional development (coaching and train- 
ing) topics. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 


improvements or a grant that required Iowa QRS participation or achieving a minimum 
rating. Nonfinancial incentives were less common at the program level: 16 percent of pro- 
grams received free or discounted equipment and materials in the previous year. 

A higher percentage of centers (79 percent) than of homes (70 percent) received a program- 
level financial or nonfinancial incentive in the previous year (see table E5 in appendix E). 
The percentage was lowest among programs with a rating of 1, and lower rated programs 
typically received nonfinancial incentives, while higher rated programs typically received 
financial incentives. 

Financial support for education or training costs and release time were the most 
common quality improvement supports received by program staff. Across all programs 
the most common support received by staff was financial support for education or train- 
ing costs (28 percent of programs), followed by release time during regular work hours 
(25 percent of programs; figure 6; see also table E6 in appendix E). There were large differ- 
ences in the percentages of centers and homes in which staff received supports, including 
financial support for education or training costs, release time during regular work hours, 
flexible schedules, transportation reimbursement, and a raise or salary adjustment based 
on completion of a degree or credentialing program. However, the absence of staff other 
than the registered provider in homes probably means that fewer formal supports would be 
offered to (or needed by) homes. There was no relationship between the programs’ initial 
QRS rating and the supports for quality improvement received by staff. 
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Figure 6. Among programs participating in the Iowa Quality Rating System, financial 
support for education or training costs and release time were the most commonly 
reported quality improvement supports received by program staff, fall 2014 


Percent of programs with staff receiving each type of support 
100 


75 - 


50 - 





Financial Release time 

support for during regular 

education or work hours 
training costs 


Flexible Books and 
schedules materials 


Transportation T.E.A.C.H. 
reimbursement scholarship 


Raise based Cash awards 
on completion to staff for 
of degree course or degree 
completion 


T.E.A.C.H. is Teacher Education and Compensation Helps. 

Note: n = 373. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 


There were large 
differences in 
the percentages 
of centers and 
homes in which 
staff received 
supports, including 
financial support 
for education or 
training costs, 
release time during 
regular work hours, 
flexible schedules, 
transportation 
reimbursement, 
and a raise or 
salary adjustment 
based on 
completion of 
a degree or 
credentialing 
program 


Among barriers to quality improvement efforts, logistical factors such as distance to 
training sites and availability of needed training were reported more often than program - 
related factors, such as staff turnover. The survey asked whether certain logistical or pro- 
gram-related barriers had no effect, a small effect, a medium effect, or a large effect on quality 
improvement efforts at the program level. At least 63 percent of survey respondents indicated 
that logistical barriers (especially distance and travel time, training schedule, and cost) had a 
medium or large effect on staff participation in quality improvement efforts (figure 7). Fewer 
than 55 percent of programs reported that program-related barriers (especially staff turnover 
and management support) had a medium or large effect. These responses on management 
support may be biased because most survey respondents were managers. Managers may 
underreport the effect of management support on quality improvement activities. 

Centers reported higher levels of barriers than did homes, but there was no statistically 
significant relationship between initial Iowa QRS rating and reported barriers (figure 8; see 
also table E7 in appendix E). 
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Figure 7. A high percentage of programs participating in the Iowa Quality Rating 
System reported large effects of logistical barriers on staff participation in 
professional development and other improvement activities, fall 2014 
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Note: n = 357-370. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 


Figure 8. Among programs participating in the Iowa Quality Rating System, the average 
level of barriers to quality improvement activities was significantly higher among 
centers than among homes, but did not differ significantly by rating level, fall 2014 


Average level of barriers 
3 -| 



Centers Homes 1 2 3 4 5 


Program type*** Iowa Quality Rating System rating level 

*** Statistically significant at p < .001. 

Note: n = 361. Levels of barriers are measured on a four-point scale: no effect (1), small effect (2), medium 
effect (3), or large effect (4). Iowa Quality Rating System ratings range from 1 (lowest) to 5 (highest), and the 
criteria for each rating differ for centers and homes; see box 3, appendix B, and table CIO in appendix C. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 


13 


Information from the Iowa Quality Rating System rating process was the most commonly reported 
source that programs used to determine their professional development needs 


Three-quarters of survey respondents (75 percent) reported using information from the 
Iowa QRS rating process to determine professional development needs (figure 9; see also 
table E8 in appendix E). Approximately half the respondents reported using recommenda- 
tions from consultants or coaches (52 percent) and quality improvement plans (49 percent) 
to determine professional development needs; other sources of information were reported 
less commonly. 6 


Centers reported using more varied types of information to determine their professional 
development needs than did homes (see table E8 in appendix E), although homes with no 
staff other than the registered provider would not need to rely on all sources of informa- 
tion measured in the survey. The percentage of programs using information from the Iowa 
QRS rating process, recommendations from consultants or coaches, quality improvement 
plans, parent surveys, and classroom observation data differed significantly by Iowa QRS 
rating; the highest percentages were among programs with a rating of 5. 


Figure 9. Information from the Iowa Quality Rating System rating process was the 
most commonly reported source that participating programs used to determine 
their professional development needs, fall 2014 

Information from QRS rating process 
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to determine 
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development 
needs; other 
sources of 
information 
were reported 
less commonly 


Note: n = 373. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 
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Nearly all programs (98 percent) rated twice either maintained the same rating or received a higher rating 


Programs participating in Iowa QRS are re-rated every two years, and ratings tended to 
increase among programs that receive a second rating (figure 10; see also table E9 in apperv 
dix E). About half the programs (52 percent; 90 of 172 programs) that were assessed twice 
obtained the same rating the second time, and 45 percent (78 of 172 programs) obtained 
a higher rating the second time. Only 2 percent of programs (4 of 172 programs) received 
lower scores at their second assessment. Rating increases were more common among pro- 
grams that started at lower ratings and were less common among programs that started at 
higher ratings. 7 

Professional development on management topics and staff with at least 15 hours of training were 
positively related to increases in Iowa Quality Rating System rating, and professional development 
on child development topics was negatively related to rating increases 


About half 
the programs 
(52 percent; 90 
of 172 programs) 
that were assessed 
twice obtained the 
same rating the 
second time, and 
45 percent (78 
of 172 programs) 
obtained a 
higher rating the 
second time 


There were mixed results for how the quality improvement activities were related to 
increases in Iowa QRS rating. 


Programs in which managers or providers received professional development on topics 
related to program management tended to increase their Iowa Quality Rating System 
ratings. Programs in which managers or providers received professional development on 
management topics had significantly higher odds of increasing their rating (odds ratio of 
3.01) than did programs in which managers or providers did not receive that professional 
development (figure 11; see also table E12 in appendix E). The relationship holds even after 
taking into account program characteristics (including the program’s initial Iowa QRS 
rating) and other types of quality improvement activities. 


Figure 10. Just under half the programs participating in the Iowa Quality Rating 
System increased their rating at the second time point, and very few programs 
received a lower rating, 2011-15 
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Note: n = 172. Percentages do not sum to 100 because of rounding. The first rating occurred in 2011, 2012, 
or 2013, and the second rating occurred two years later. 

Source: Iowa Quality Rating System administrative data. 
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Figure 11. Among programs participating in the Iowa Quality Rating System, 
the odds of increasing the rating were higher among programs that provided 
professional development on management topics and programs that provided staff 
with 15 or more hours of training, 2011-15 


Odd s ratio 



Management Fifteen 

topics* or more 

training hours* 


Grants Continuing 
education 


Coaching Self- 

monthly assessment 
or more topics 


Classroom Nonfinancial 
practices incentives 
topics 


Training Child 

with development 

follow-up topics** 


* Significant at p < .05; ** significant at p < .01. 


Note: n = 146. This chart provides a graphic representation of the results of logistic regression model 1 
reported in table E12 in appendix E. The odds ratio bars above the center line indicate the odds of increasing 
the program's rating at the second time point if a program completed the quality improvement activity. The 
odds ratio bars below the center line are inversions of the odds ratios presented in table E12, and these bars 
indicate the odds of not increasing the program’s rating at the second time point if a program completed the 
quality improvement activity. The odds ratios were inverted so that the bars below the center line would be 
comparable in relative magnitude to the bars above the center line. 


Source: Authors' analysis of administrative data on program characteristics and Iowa QRS ratings from Iowa 
Department of Human Services and analysis of data from a survey administered in fall 2014 by the Iowa De- 
partment of Human Services, in partnership with regional Child Care Resource and Referral agencies. 


Programs that provided key staff with 15 or more training hours per year also increased 
their ratings, though such increases may partly reflect how training hours are factored 
into the Iowa Quality Rating System rating. The National Institute for Early Education 
Research recommends that key staff in early childhood education programs receive at least 
15 hours of training per year (Barnett et al, 2014). The findings of this study support this 
recommendation: programs in which key staff received an average of at least 15 hours of 
training in the previous year were more likely to increase their rating (odds ratio of 2.90) 
than were programs that did not provide such training (see figure 11; see also table El 2 in 
appendix E), holding constant program characteristics, initial Iowa QRS rating, and other 
types of quality improvement activities. 

However, the positive relationship between at least 15 hours of training and rating increas- 
es may be partly due to how the rating is calculated. Some programs obtain a higher profes- 
sional development domain score because the number of points earned in the professional 
development category of the Iowa QRS rating is partly determined by hours of training, 
and the professional development domain score is one of the criteria used to determine 
ratings 3, 4, and 5. That means that the number of training hours may be helping pro- 
grams increase their ratings simply because hours are counted directly in the rating score 


Programs in which 
key staff received 
an average of at 
least 15 hours 
of training in the 
previous year 
were more likely 
to increase their 
rating (odds ratio 
of 2.90) than were 
programs that 
did not provide 
such training 
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rather than through the impacts of the training. Analyses of the relationship between at 
least 15 hours of training and increases in the professional development domain score find 
even stronger relationships, further suggesting that this may be the case. 

Programs in which staff receive professional development in child development were 
less likely than programs in which staff did not receive professional development in 
child development to increase their Iowa Quality Rating System ratings. Although 
these analyses do not examine causal relationships, they identified this negative relation- 
ship after accounting for program characteristics, initial rating, and other types of quality 
improvement activities (see table E12 in appendix E). 

Domain score analyses suggest a limited relationship between quality improvement activ- 
ities and increases in Iowa Quality Rating System rating. Ratings are based on domain 
scores for programs eligible for a rating of 3 or higher (see box 3 and appendix B). Among the 
subset of programs with domain scores, the study examined the relationship between quality 
improvement activities and increases in domain scores for domains that are common to both 
centers and homes. The logistic regression models found relatively few significant relation- 
ships between quality improvement activities and domain score increases (table 1). However, 


Table 1. The logistic regression models showed some differences in the 
relationships between the quality improvement activities of participating programs 
and each Iowa Quality Rating System outcome, 2011-15 


Key quality improvement activity 

Improvement 
in Iowa Quality 
Rating System 
rating 
(n = 146) 

Improvement 
in health and 
safety score 
(n = 109) 

Improvement 
in environment 

score 
(n = 109) 

Improvement 
in professional 
development 
score 
(n = 109) 

Management topics 

+ * 

+ 

+ 

+ 

Fifteen or more training hours 

+ * 

+ 

- 

+ * 

Grants 

+ 

+ 

- 

+ 

Continuing education 

+ 

- 

- 

+ 

Coaching monthly or more 

+ 

- 

- 

- 

Self-assessment topics 

+ 

+ 

+ 

- 

Classroom practices topics 

- 

+ 

+ 

- 

Nonfinancial incentives 

- 

- 

- 

+ 

Training with follow-up 

- 

+ 

+ 

- 

Child development topics 

_ ** 

- 

_ * 

- 


* Significant at p < .05; ** significant at p < .01. 

+ indicates a positive relationship between the quality improvement activity (table rows) and the Iowa Quality 
Rating System (Iowa QRS) outcome (columns). 

- indicates a negative relationship between the quality improvement activity and the Iowa QRS outcome. 

Note: The table shows the direction of the relationships in logistic regression models that account for the 
listed quality improvement activities and that control for initial Iowa QRS rating, program type, public preschool 
or Head Start funding, and receipt of child care subsidies. For detailed model results, see model 1 in table 
E12 in appendix E for increases in Iowa QRS rating, model 3 in table E12 for increases in health and safety 
score, model 5 in table E12 for increases in environment score, and model 9 in table E12 for increases in 
professional development score. The results are not included for increases in family and community partner- 
ships scores because model fit statistics indicated a poor fit to the data, and the results should therefore not 
be interpreted, though they can be viewed in table E12. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 
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the five statistically significant relationships observed in these analyses provide additional 
information to help interpret the relationship between quality improvement activities and 
changes in the overall ratings (because the domain scores are part of the overall ratings for 
more highly rated programs). It may be interesting to explore these issues in future research. 

An average of 15 or more training hours among key staff was significantly and positively 
related to increases in Iowa QRS ratings and also to increases in the professional deveh 
opment domain score but not in other domain scores. This underlines the possibility that 
the significant relationship between 15 or more training hours and increases in Iowa QRS 
ratings may be driven by the way training hours are factored in to scores on the profes- 
sional development domain. The provision of professional development on child develop- 
ment topics to teachers or providers was significantly and negatively related to increases 
in Iowa QRS ratings and to increases in environment scores. The provision to managers 
or providers of professional development on child development topics was not significantly 
related to increases in other domain scores. The provision of professional development on 
management topics was positively associated with Iowa QRS rating increases but not signifi- 
cantly related to any domain score increases. There were no other significant relationships 
between quality improvement supports and increases in domain scores or Iowa QRS ratings. 

Implications of the study findings 


The descriptive findings of this study provide an overview of the landscape of quality 
improvement activities among early childhood education programs participating in the 
Iowa QRS. The results suggest that in-person training is the most common type of quality 
improvement activity in Iowa, perhaps partly because annual in-person training hours are 
required to maintain home provider registration or center licensing. While staff in almost 
all programs completed in-person training, staff in approximately half the programs partic- 
ipated in web-based training. However, many survey respondents identified long distance 
and travel time as barriers to participation in quality improvement activities such as train- 
ing. Members of the Iowa Quality Rating System Oversight Committee suggested that 
programs could increase overall staff training hours by encouraging staff to use web-based 
training after meeting the minimum requirements for in-person training. Because web- 
based training requires access to appropriate technology, the state may wish to examine 
the accessibility of web-based training options for staff in programs across Iowa. 

Although staff in almost all programs participated in training, fewer than half the pro- 
grams met the study team’s adaptation of the National Institute for Early Education 
Research benchmark of providing an average of at least 15 hours of training per year for 
key staff (Barnett et al, 2014). The regression analyses suggest that meeting the 15-hour 
benchmark is associated with a higher likelihood of increasing the Iowa QRS rating, 
although the analyses do not provide information on whether additional training leads 
to increases in rating (the increases could be caused by other factors not accounted for 
in the study). Furthermore, the number of points earned in the professional development 
category of the Iowa QRS rating is partly determined by hours of training. That means 
that the number of training hours may be helping programs increase their ratings simply 
because hours are counted directly in the rating score rather than through the impacts of 
the training. The state may wish to investigate the value of encouraging training hours 
beyond the minimum requirements for licensing or registration and the potential role of 
web-based training in supporting more training among program staff. 
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About two'thirds of programs reported that staff received coaching. Coaching provided 
on an ongoing basis (once a month or more) allows programs to engage with the coach 
in continuous quality improvement, and less frequent consultation with a coach can help 
programs address a particular issue or meet a particular need. Only 15 percent of programs 
reported that staff received ongoing coaching. The regression analysis did not find a sig- 
nificant relationship between ongoing coaching and increases in Iowa QRS ratings, but 
coaching may positively affect the classroom practices of coaching participants in ways 
that are not reflected in program-level Iowa QRS ratings. Because other research has found 
positive relationships between coaching and classroom quality (Isner et al., 2011; Muen- 
chow et al., 2013; Neuman & Cunningham, 2009; Tout et al., 2011), the state may wish to 
investigate how coaching affects classroom quality. 

The findings indicate that staff in 92 percent of the programs received professional devel- 
opment on health and safety topics and that staff in 85 percent of the programs received 
professional development on child development topics. Though these shares are high, 
members of the Iowa Quality Rating System Oversight Committee expressed surprise 
that participation in training on these topics is not universal among programs, given the 
emphasis on these topics by early childhood funding agencies within the state. The regres- 
sion results find a counterintuitive negative relationship between teacher professional 
development in child development topics and increases in Iowa QRS ratings. This result 
should be interpreted with the same caution as the other regression-based results because 
the results may be explained by other factors. For example, the programs that did not focus 
on child development topics may have staff with prior expertise in child development, 
which might also help drive rating increases. The study did not have data on prior child 
development training or on the expertise of teachers and so could not examine this pos- 
sibility empirically. Domain score analyses suggest that the negative relationship between 
professional development in child development topics and Iowa QRS rating increases is 
reflected in an even stronger negative relationship with environment score increases. The 
state may wish to conduct additional research to understand this finding more clearly. 


Staff in about half the programs received professional development in self-assessment and 
management topics. The regression analysis findings indicate that professional develop- 
ment among managers on management topics was positively correlated with Iowa QRS 
rating increases. This positive relationship suggests that the state may wish to conduct 
further research to examine whether the management training is leading to improvements 
in program quality or whether other factors explain the observed relationships. Should the 
state decide to offer more management training opportunities, it would be helpful to pilot 
the offerings within a subset of programs to study the impact on QRS ratings. 

Staff in about one-third of programs pursued a degree or credential in the year before the 
survey, a fairly small percentage. Not all early childhood education program staff require 
continuing education, particularly if they have already completed a degree appropriate for 
the job, but the state may wish to investigate access to continuing education among early 
childhood education program staff. 


The state may 
wish to conduct 
further research to 
examine whether 
the management 
training is leading 
to improvements 
in program quality 
or whether other 
factors explain 
the observed 
relationships 


Centers engaged in some types of quality improvement activities more often than did 
homes, including web-based training, monthly coaching, continuing education, and new 
staff orientation and mentoring. Home providers participated in a higher number of annual 
training hours, on average, than did centers, but this difference may be due to the larger 
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number of training hours required for home providers or to the availability of fewer staff to 
send for training. Homes offered fewer supports for staff participation in professional devel- 
opment and continuing education. Many homes have only one lead provider, and so they 
may not feel the need to offer formal supports for professional development or education. 
Yet even homes with assistants make little use of quality improvement activities such as 
new staff orientation. It could be that home providers find it difficult to participate in activ- 
ities that require absence from the program if they have no staff to cover for staff absent for 
training. The state may wish to consider ways to increase access to quality improvement 
activities among homes, such as developing quality improvement activities and materials 
targeted specifically to homes, developing coaching supports that count toward annual 
training hours, or providing web-based or video coaching that does not require travel. 

Programs with an initial rating of 5 were more likely than programs that received lower 
ratings to participate in some quality improvement activities, such as monthly coaching 
and continuing education. Other quality improvement activities increased steadily with 
each higher Iowa QRS rating, such as new staff orientation and mentoring. It is not sur- 
prising that programs with high ratings were most likely to engage in quality improve- 
ment activities. The state may wish to consider ways to encourage participation in quality 
improvement activities among programs at lower Iowa QRS ratings, the programs that are 
most in need of improvement. For example, the state could offer incentives to encourage 
programs with a rating of 1 or 2 to participate in quality improvement. 


Among the staff supports for quality improvement activities measured in the survey, even 
the most common ones were used in less than 30 percent of programs. Many of these 
supports could lower the logistical barriers to quality improvement reported by many pro- 
grams; for example, by providing transportation reimbursement, flexible schedules, and 
release time. The state may wish to consider ways to expand the availability of supports to 
help early childhood education program staff participate in quality improvement activities. 

Information from the Iowa QRS rating process was used widely by programs to determine 
their professional development needs, suggesting that participation in the Iowa QRS is 
useful to programs in identifying needs and planning quality improvements. 


The state may 
wish to consider 
ways to expand 
the availability of 
supports to help 
early childhood 
education program 
staff participate 
in quality 
improvement 
activities 


Limitations of the study 


This study has a number of limitations that should be considered when reviewing the 
study’s findings: 

• All information about quality improvement activities was self-reported by pro- 
grams that participated in a survey administered by the Iowa Department of 
Human Services, and the accuracy of the self-reported information has not been 
independently verified. 

• The information about quality improvement activities was collected from a subset 
of randomly selected programs that completed the survey and may not represent 
the activities completed by all programs participating in the Iowa QRS. 

• The survey asked respondents about quality improvement activities conducted 
during the previous 12 months. Programs may have done other quality improve- 
ment activities during the other 12 months of the QRS rating cycle that may not 
be reflected in the findings. 
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• The analyses employed in this study are intended to describe the use of quality 
improvement activities and the relationships between those activities and increas- 
es in QRS ratings. However, the study was not designed to support causal inference 
about the impacts of specific quality improvement activities on program quality. 
To determine whether the activities have effects on program quality, a randomized 
controlled trial would need to be performed. 

• The regression analyses examine how quality improvement activities relate to 
Iowa QRS ratings and domain scores rather than to direct child outcomes, and 
relationships between these activities and Iowa QRS ratings may not reflect how 
quality improvement activities affect child learning and development. 
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Appendix A. Supplemental literature review 


This appendix supplements the brief literature review included in the main body of the report. 

The literature suggests that quality improvement supports and activities among early childhood 
education programs have the potential to improve children’s early learning experiences 

Quality improvement activities include training, coaching, mentoring, and continuing 
education. Quality improvement supports include financial incentives, such as tiered reim- 
bursements and rating bonuses, and nonfinancial incentives, such as access to materials 
and equipment. Box A1 summarizes the research (mostly correlational) on each type of 
quality improvement activity and early childhood education program outcome, outside the 
context of quality rating and improvement systems (QRISs). 

The literature suggests that quality improvement activities have the potential to improve 
program quality and possibly also child outcomes, although less is known about how varia- 
tion in the specific quality improvement activities may influence the effectiveness of these 
activities. In addition, some research suggests that comprehensive interventions combin- 
ing training and coaching may be more effective at improving classroom practices and 
child outcomes than either type of quality improvement support alone (Landry, Anthony, 
Swank, & Monseque-Bailey, 2009; Landry, Swank, Anthony, & Assel, 2011). However, 
these studies examine quality improvement activities outside the context of QRISs, and 
programs participating in a QRIS may use quality improvement activities differently than 
programs that do not participate in QRISs. Furthermore, these studies typically focus on 
how quality improvement activities affect interactions between staff and children. QRISs 
are multidimensional in nature, and the ratings typically combine data on program and 
staff characteristics, the program environment, and in some cases interactions between 
staff and children. For this reason, the findings of the literature reviewed here do not nee- 
essarily generalize to the QRIS setting and do not inform states about the relationship 
between quality improvement activities and changes in QRIS ratings. 

To date, few studies have described how programs in quality rating and improvement systems use 
quality improvement supports and activities or how these supports and activities relate to changes 
in ratings 

A handful of studies describe the types of quality improvement activities available to 
or used by programs participating in a QRIS. Programs participating in a QRIS tend 
to receive more professional development than programs that do not (Isner et al., 2011; 
Muenchow et al., 2013; Smith, Robbins, Schneider, Kreader, & Ong, 2012; Smith, Schnei- 
der, & Kreader, 2010; Tout et al., 2011). Conducting case studies with four local and pilot 
QRISs, Isner and colleagues (2011) found that QRIS administrators and directors of quality 
improvement services reported that QRIS programs emphasized on-site professional devel- 
opment processes relatively more, including in the duration and intensity of the processes. 
In addition, survey data gathered from technical assistance providers in 17 states indicate 
that less highly rated programs participating in QRISs were less likely to access on-site 
technical assistance supports, although half the technical assistance providers reported 
that they targeted quality assistance visits to programs with greater needs (Smith et al., 
2010, 2012). Across these studies, small sample sizes, the reliance on qualitative interview 
data, and the concerns inherent in self-reported data limit the generalizability. 
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Box Al. There is a sizable body of literature on quality improvement supports and activities and early 
childhood education program outcomes 

Training refers to professional development administered to a group of early childhood education program staff, 
conducted through workshops or professional meetings that do not accrue higher education credits (Zaslow, Tout, 
Halle, Whittaker, & Lavelle, 2010) Some studies find that training, especially if it is intensive, content specific, and 
focused on child-teacher interactions, is associated with improvements in classroom practices, observed program 
quality, and caregiver competency (for example, Fukkink & Lont, 2007; Hamre et al., 2012; Raikes et al., 2006; 
Zaslow et al., 2010). However, other studies find no impact of training alone, suggesting that training is most effec- 
tive if it includes follow-up supports, such as on-site technical assistance or coaching (Tout, Epstein, Soli, & Lowe, 
2015). In some research, training had positive effects on some teaching practices but not on others, suggesting 
that training alone may not be sufficient to change the practice in some areas, although the training program did 
have a positive effect on child language outcomes (Piasta et al., 2012). A large experimental study of the Program 
for Infant/Toddler Care, an on-site training program that also includes some coaching, found no impacts on global 
program quality, adult-child interactions, or child outcomes, although the study also found that staff in many pro- 
grams did not complete the full 56 hours of training and thus did not receive the full program (Weinstock et al., 
2012 ). 

Coaching (as well as mentoring and on-site technical assistance) refers to individualized task-oriented professional 
development strategies typically conducted one-on-one with staff, often in the program setting. Studies suggest 
that coaching is associated with higher observed program quality and improved classroom practices, but there 
is mixed evidence on the relationship between coaching and child developmental outcomes in language, literacy, 
math, and behavior (Aikens & Akers, 2011; Bryant et al., 2009; Hamre, Pianta, Mashburn, & Downer, 2012; Isner 
et al., 2011; Muenchow et al., 2013; Neuman & Cunningham, 2009; Tout et al., 2011; Tout et al., 2015; Wasik & 
Hindman, 2011). Although some studies have examined the mechanisms through which coaching may be most 
effective and suggest better outcomes from more sustained coaching (Pianta et al., 2014; Wasik & Hindman, 2011), 
additional research is needed on specific elements of coaching (such as type, duration, and frequency) and on the 
characteristics of the coaches to understand more clearly the most effective way to implement coaching (Aikens & 
Akers, 2011; Isner et al., 2011; Muenchow et al., 2013). 

Continuing education refers to the pursuit of credentials, credits, or degrees by employed early childhood edu- 
cation program staff. Findings are mixed on the relationship between educational attainment or credentials and 
program quality. Some studies find no association, and others find positive associations; there is little evidence of 
a relationship between teacher education level and child outcomes (Early et al., 2006; Tout, Zaslow, & Berry, 2001, 
Zaslow et al., 2010). Some evidence suggests that the relationship between education level and program quality 
may vary according to the characteristics of the degree program and the characteristics of the early childhood edu- 
cation program, but additional research is needed to unpack how variation within specific types of training and other 
professional development is related to improvements in quality (Vu, Jeon, & Howes, 2008; Zaslow et al., 2010). 

Incentives for quality improvement include financial incentives (which encompass monetary awards such as schol- 
arships, bonuses, or tiered reimbursements) and nonfinancial incentives (which include nonmonetary rewards such 
as equipment and materials). There is little research examining financial incentives and how they relate to program 
quality or child outcomes (Mitchell, 2012), but some studies suggest that financial incentives facilitate accreditation 
and are associated with developmental^ appropriate practices and beliefs (Adams, Bierbrauer, Edie, Riley, & Roach, 
2003; Cassidy, Buell, Pugh-Hoese, & Russell, 1995; Gormley & Lucas, 2000). Research is needed especially on 
nonfinancial incentives and quality. 
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A few studies have performed descriptive analyses employing administrative data to 
examine how programs use quality improvement activities within QRISs. In California, 
Muenchow et al. (2013) found that, across local and statewide QRISs, participating pro- 
grams consistently invested in coaching and mentoring, while training and formal edu- 
cation supports also enjoyed widespread use as quality improvement strategies. Tout et al. 
(2011) found that fully rated programs participating in Minnesota’s pilot QRIS most com- 
monly accessed quality improvement supports intended to improve QRIS-related docu- 
mentation and their performance in Environment Rating Scale observations. In addition, 
preliminary analyses conducted by Yazejian and Iruka (2015) described the distribution of 
technical assistance hours, awards, and scholarships to QRIS-participating programs in 
Miami-Dade County, Florida. They found that programs received 5 hours a month of tech- 
nical assistance, on average. Findings from these studies must be interpreted cautiously 
given that states often face limitations in their ability to collect reliable administrative 
data that documents how programs access and participate in quality improvement efforts. 
This restricts a state’s ability to understand how quality improvement efforts are linked to 
QRIS program ratings (for example, because of problems in linking data across systems). 
Furthermore, states do not have the resources and systems in place to verify the accura- 
cy of administrative data. Therefore, states lack sufficient information about the specific 
quality improvement supports and activities in which programs in the QRIS engage. 

Research focused specifically on the relationship between quality improvement activi- 
ties and QRIS outcomes is limited. Some QRIS research funded through state Race to 
the Top-Early Learning Challenge awards are examining relationships between quality 
improvement activities within the context of QRIS and various outcomes, including QRIS 
rating changes, program quality outcomes, and child developmental outcomes. However, 
this research is still under way and has not yet been published in most states. Other studies 
provide limited information about the relationship between quality improvement activities 
and rating increases. 

Analyzing administrative data for 412 center- and home-based programs in Miami’s QRIS, 
Yazejian and Iruka (2015) studied the relationship between three types of quality improve- 
ment supports (on-site technical assistance, nonfinancial incentives in the form of mate- 
rials and equipment, and scholarships for credentials or degrees) and subsequent QRIS 
ratings. Among centers, the authors found that only scholarships had a small positive rela- 
tionship with subsequent QRIS ratings and that the relationship was slightly larger among 
programs that had been in the QRIS for less time. Among homes, there were no signif- 
icant relationships between quality improvement supports and QRIS ratings, but home 
programs that had been in the QRIS for less time showed a slightly larger relationship 
between scholarships and QRIS ratings. The study found that quality improvement sup- 
ports are related only weakly to ratings improvements, although the use of administrative 
data on a small subset of quality improvement activities curbs the extent to which conclu- 
sions can be drawn from this research. 

Other studies have examined the effects of specific combinations of quality improvement 
activities and supports that are provided to programs in a QRIS. One study examined 
the QRIS rating outcomes among programs participating in Pennsylvania’s Success by 
6 program, an intensive quality improvement program that includes intensive coaching, 
program improvement funds, and other supports designed to help the programs reach a 
rating of 3. Using a matched comparison design with propensity score matching, the study 
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compared rating increases among programs with a rating of 2 that did and did not vol- 
untarily participate in the Success by 6 quality improvement program. It found substan- 
tially higher rates of rating increases among programs participating in the Success by 6 
program (Warner-Richter et al., 2016). Another study used a program-level random assign- 
ment design to examine the impacts of the quality improvement supports offered as part of 
Washington’s Seeds to Success QRIS — including quality improvement grants, funding for 
professional development, and coaching using the Consultative Coaching Program model 
— on program quality and QRIS rating outcomes. The study found positive impacts on 
observed quality, but no impact on the overall QRIS rating (Boiler et al., 2015). These 
studies provide useful information about the potential benefits of quality improvement 
supports for programs in QRISs, but the studies were not designed to capture the specific 
aspects of the quality improvement interventions that were most effective, and the results 
cannot be generalized to other QRISs offering quality improvement supports that do not 
follow the same program model. 

Additional research is needed on quality improvement activities and QRIS outcomes. 
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Appendix B. Iowa Quality Rating System rating criteria 


This appendix includes the Iowa Quality Rating System (QRS) rating guidelines for centers 
and homes, based on documents provided by the Iowa Quality Rating System Oversight 
Committee. 

Guidelines for centers include criteria for five ratings 

The Iowa QRS rating criteria for centers are presented below. 


Rating of 1 and 2 


1 

• Full licensing OR a provisional license with no action to revoke or deny OR operates under the authority of 
an accredited school district or nonpublic school 

2 

• Full licensing only with no action to revoke or deny OR the program operates under authority of an 
accredited school district or nonpublic school 

If eligible, participation in federal food program (Child and Adult Care Food Program) 

• Each room has at all times at least one staff member present who has completed mandatory reporting 
of child abuse, universal precautions and infectious disease control, cardiopulmonary resuscitation, and 
first aid 

• Basic orientation for all staff prior to beginning work 

Director and staff perform self-assessments of each individual's skills and one of the center overall 


Rating of 3-5 

The amount of points from the menu below is required to receive a rating of 3 5: 


3 17-26 points 

4 27-33 points 

5 Minimum of 34 points and minimum Environment Rating Scale (ERS) assessment score of 5.0 
in each assessed room. The assessment must be completed by Iowa State University. 
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Credential — programs may earn a maximum of 5 points 

Center director has one of the following: 

• Valid National Administrator Credential (NAC) 

• Valid Aim4Excellence credential 

• Valid license as a preK principal issued by the Board of Educational Examiners 

• OR 

Points 

• Staff has completed the Head Start Management Acceleration Program (MAP) 

5 

Education and experience — programs may earn a maximum of 25 points 

Each staff member shall indicate the highest applicable education and experience 
qualification, and the total points of all staff will be divided by the number of staff. 

Points 

15 hours of annual approved training beyond regulatory requirements 

2 

30 hours of annual approved training beyond regulatory requirements and at least 5 years 
of experience working in a child care facility or a program operating under the authority of an 
accredited school district or nonpublic school 

4 

At least 9 college credit hours in education specific to age group for whom care is provided 

5 

Iowa Board of Educational Examiners paraeducator certificate at level 2, early childhood, plus 

2 years of experience in early childhood education under the supervision of a licensed early 

childhood teacher 

6 

Child Development Associate (CDA) credential 

6 

Apprenticeship certificate 

7 

1-year diploma in early childhood education 

8 

An associate's degree in education specific to age group for whom care is provided 

10 

A bachelor's degree in education specific to age group for whom care is provided 

20 

A master’s degree in education specific to age group for whom care is provided 

25 

Health and safety 

Maximum points = 19 

Points 

Director, assistant director, or on-site supervisor completes 3-semester-hour health, safety, 
and nutrition class through community or 4-year college (must have been completed within the 
past 5 years) 

5 

Other approved health and safety training option (must have been completed within the past 2 
years) 

2 

Development and implementation of an emergency preparedness plan 

2 

Development and implementation of enhanced health and safety policies 

2 

Completion of injury prevention checklist with child care nurse consultant 

Visit completed — 1 point 

Starting process of making recommended corrections — 2 points 

All corrections completed — 3 points 

1-3 

Completion of child record review with child care nurse consultant 

Visit completed — 1 point 

Development of a plan of action to secure health services for children — 2 points 

1-2 

Completion of health and safety assessment with child care nurse consultant 

Visit completed — 1 point 

Development of a plan of action to correct deficiencies — 2 points 

All corrections completed — 3 points 

1-3 
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Environment 

Maximum points = 27 


Training and self-assessment — programs may earn a maximum of 9 points 

Points 

Center director or assistant director completes Iowa State University Extension ERS training 
appropriate to the ages of children in care 

2 

After completing ERS training, the facility director or assistant director completes a self- 
assessment and score sheet for at least one-third of the facility's classrooms, including at 
least one classroom in each age group served by the facility. 

2 

After completing ERS training, the facility director or assistant director completes a child care 
center improvement plan for each room in which the self-assessment was completed. 

2 

After completing Iowa Quality Preschool Program Standards training, center director or 
assistant director completes Iowa Quality Preschool Program Standards self-assessment and 
develops quality improvement plan 

3 

Enhanced ratios — programs may earn a maximum of 3 points 

Points 

Meets National Association for the Education of Young Children or National AfterSchool Association 
standards for group/class size appropriate to setting (only for programs not accredited by 

National Association for the Education of Young Children or National AfterSchool Association) 

3 

Accreditation preparation — programs may earn a maximum of 5 points 

Points 

Accreditation self-assessment approved by National Association for the Education of Young 
Children (only for programs not accredited by National Association for the Education of Young 
Children) 

5 

Accreditation — programs may earn a maximum of 18 points. Programs may receive points for 
one of the three options below: 

Points 

Program is verified by Iowa Quality Preschool Program Standards 

5 

Compliance with Head Start Program Performance Standards 

6 

Accreditation by National Association for the Education of Young Children, Council on 
Accreditation (after-school or 8th edition standards), or National AfterSchool Association 

18 


Family and community partnerships 

Maximum points = 8 

Points 

Program or director is a member of a professional organization specific to age group for whom 
care is provided 

1 

Orientation provided for new parents 

1 

Annual conferences are held with parents 

1 

At least one group parent meeting is held annually 

1 

Parent advisory board meets quarterly 

2 

Annual parent surveys are collected, and results are used to inform program practices 

2 


Leadership/administration 

Maximum points = 7 

Points 

All staff receive yearly written evaluation 

2 

Development and annual updating of an overall center improvement plan 

1 

All staff have completed professional development plans with the center's overall skill needs 
in mind 

1 

All staff who have direct contact with children complete the Iowa State University Extension 

New Staff Orientation training within 4 months of starting employment 

3 


ERS is Environment Rating Scale. 

Source: Documents provided by the Iowa Quality Rating System Oversight Committee. 
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Guidelines for homes include criteria for five ratings 


The Iowa QRS rating criteria for homes are presented in this section. 


Rating of 1 and 2 


1 

• Provider is registered with the Department of Human Services. 

2 

The provider: 

Is registered with the Department of Human Services. 

• Completes and maintains ChildNet certification. 

• Participates in federal food program (Child and Adult Care Food Program). 

• Completes a self-assessment of own professional development. 

• Writes professional development plan. 


Rating of 3-5 

The amount of points from the menu below is required to receive a rating of 3 5: 
(at least one point must be earned from each category) 


3 14-18 points 

4 19-24 points 

5 Minimum of 25 and FCCERS-R assessment score of 5.0 or greater. The assessment must be 
completed by Iowa State University. 


Professional development 

Maximum points = 34 


Experience and training — programs may earn a maximum of 4 points 

Points 

At least 2 years of child care experience and 10 hours of additional approved training per year 
beyond regulatory requirements 

2 

At least 5 years of child care experience and 20 hours of additional approved training per year 
beyond regulatory requirements 

4 

Additional professional development — programs may earn a maximum of 5 points 

Points 

Completion of Positive Behavioral Interventions and Supports training, 
modules 1 and 2 (minimum of 12 hours training) 

2 

Completion of Program for Infant and Toddler Care, modules 1-4 

3 

Education — programs may earn a maximum of 25 points — 
points will be awarded for only one of the following criteria: 

Points 

At least 9 college credit hours in education specific to age group for whom care is provided 

5 

Child Development Associate (CDA) credential 

6 

Apprenticeship certificate 

7 

1-year diploma in early childhood education 

8 

An associate's degree in education specific to age group for whom care is provided 

10 

A bachelor's degree in education specific to age group for whom care is provided 

20 

A master’s degree in education specific to age group for whom care is provided 

25 
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Health and safety 

Maximum points = 19 


Points 

Completion of 3-semester-hour health, safety, and nutrition class through community or 4-year 
college (must have been completed within the past 5 years) 

5 

Other approved health and safety training option (must have been completed within the past 

2 years) 

2 

Development and implementation of an emergency preparedness plan 

2 

Development and implementation of enhanced health and safety policies 

2 

Completion of injury prevention checklist with child care nurse consultant 

Visit completed — 1 point. 

Starting process of making recommended corrections — 2 points. 

All corrections completed — 3 points. 

1-3 

Completion of child record review with child care nurse consultant 

Visit completed — 1 point. 

Development of a plan of action to secure health services for children — 2 points. 

1-2 

Completion of health and safety assessment with child care nurse consultant 

Visit completed — 1 point. 

Development of a plan of action to correct deficiencies — 2 points. 

All corrections completed — 3 points. 

1-3 


Environment 

Maximum points = 23 

Points 

Completion of Iowa State University Extension training on FCCERS-R 

2 

After completing approved training on how to use the FCCERS-R, completes self-assessment 
and score sheet by using FCCERS-R 

2 

After completing approved training on how to use the FCCERS-R, completes child development 
home improvement plan based on FCCERS-R self-assessment 

2 

No more than two children under age 2 are in care at any one time, and no more than six 
children total are in care at any one time, including the provider's own children under school 
age 

2 

Accreditation by the National Association for Family Child Care 

15 


Family and community partnerships 

Maximum points = 6 

Points 

Membership in a professional organization specific to age group for whom care is provided 

1 

Orientation provided for new parents 

1 

Annual conferences are held with parents. 

1 

At least one group parent meeting is held annually. 

1 

Annual parent surveys are collected, and results are used to inform program practices. 

2 


FCCERS-R is Family Child Care Environment Rating Scale — Revised. 

Source: Documents provided by the Iowa Quality Rating System Oversight Committee. 
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Appendix C. Data and methodology 


This appendix provides detailed information about the study data and methodology. 

Sampling strategy 

For the descriptive analyses (research questions 1, 2, and 3), the sample consisted of all 
respondents to the quality improvement survey administered by the Iowa Department 
of Human Services (Iowa DHS), in partnership with regional Child Care Resource and 
Referral (CCRR) agencies, in October and November 2014- The survey was sent to 600 
randomly selected early childhood education programs participating in the Iowa Quality 
Rating System (Iowa QRS); 390 programs responded to the survey. For the regression 
analyses (research questions 5 and 6), the sample consisted of the 146 programs that 
responded to the entire survey, received a new Iowa QRS rating during the period January 
2014 to March 2015, and had at least one previous Iowa QRS rating. 

The sampling frame for the survey consisted of the 972 programs in Iowa (including 508 
family child care homes and 464 centers) with active Iowa QRS ratings in September 2014 
(shortly before survey administration) that had been participating in the system since 
at least December 2013. The sampling frame excluded an additional 272 programs with 
active ratings in September 2014 that had entered the system in 2014 because the survey 
was designed to measure quality improvement activities across the course of the previous 
12 months. The study team constructed the sampling frame by using state administra- 
tive data on Iowa QRS ratings across multiple time points. The sampling frame data files 
include a unique study identifier, program name, program address, license or registration 
numbers, program email address, county, region of the state (1-5), program type, data from 
multiple time points on the Iowa QRS domain and total scores (points earned based on 
Iowa QRS standards), Iowa QRS rating, date of rating application and expiration date, and 
amount of bonus award based on QRS rating. 

To determine the total sample size of 600 programs for the survey invitations, the study 
team first conducted power analyses to estimate the number of programs needed for 
the study’s regression analyses (360) and then increased that number assuming a survey 
response rate of approximately 60 percent. The random sample of 600 programs was strati- 
fied by program type (center and family child care) and Iowa QRS rating (1, 2, 3, 4, and 5) 
to increase the precision of the survey estimates. The study team determined the number 
of programs to select from each stratum by calculating the percentage of programs in each 
stratum in the full population of 972 eligible programs and applying these percentages to 
the desired sample size of 600. The number of programs selected in each stratum is illus- 
trated in table Cl; a total of 600 programs were selected. 

The Iowa DHS and the Iowa CCRR agencies received 390 survey responses, a response 
rate of 65 percent. To gauge the likelihood that the survey results are representative of 
all programs participating in the Iowa QRS, the Regional Educational Laboratory (REL) 
Midwest study team conducted a nonresponse bias analysis comparing characteristics 
of the 390 survey respondents with the full sample of 600 programs that were randomly 
selected for the survey. The distribution of survey respondents across each of the sam- 
pling strata is similar to the distribution in the full sample of programs selected for the 
survey and does not differ significantly from the expected distribution (table C2). These 
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Table Cl. Number of programs participating in the Iowa Quality Rating System that 
were randomly selected for the survey, by program type and rating, September 


2014 




Iowa QRS rating 




Program type 

1 

2 

3 

4 

5 

All levels 

Centers 

3 

58 

64 

133 

28 

286 

Homes 

18 

149 

52 

86 

9 

314 

All programs 

21 

207 

116 

219 

37 

600 


QRS is Iowa Quality Rating System. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 


Table C2. Percentage of programs participating in the Iowa Quality Rating System 
among survey respondents and full sample of programs selected for the survey by 
program type and rating, September 2014 


Iowa Quality 
Rating System 

Survey respondents 
(n = 390) 

Full sample selected for survey 
(n = 600) 

rating 

Home 

Center 

Total 

Home 

Center 

Total 

1 

1.79 

0.77 

2.56 

3.00 

0.50 

3.50 

2 

22.05 

9.49 

31.54 

24.83 

9.67 

34.50 

3 

10.00 

10.77 

20.77 

8.67 

10.67 

19.33 

4 

17.69 

20.00 

37.69 

14.33 

22.17 

36.50 

5 

1.79 

5.64 

7.44 

1.50 

4.67 

6.17 

Total 

53.33 

46.67 

100.00 

52.17 

47.83 

100.00 


Note: The y 2 analyses comparing the distribution of survey respondents in the sampling strata with the ex- 
pected number of respondents based on the distribution of the full sample showed no significant differences 
(y 2 = 9.398, p = .4014). 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 


data provide evidence that the programs that responded to the survey are representative of 
programs across the state for the two program characteristics prioritized for sampling. 

Additional analyses compare other characteristics of survey respondents to the full 
sample of randomly selected programs invited to participate in the survey and also to 
nonrespondents in the randomly selected sample (tables C3 and C4). Response bias is 
assessed according to the raw percentage point difference between all randomly selected 
programs and responding programs and the percent relative difference, which divides the 
raw percentage point difference by the percentage among all randomly selected programs 
(Garrison-Mogren, Bullis, & Falls, 2007). In addition, the study team tested the hypothesis 
of independence between each characteristic and participation status using a chi-square 
statistic at the 5 percent level. These analyses employed data on program characteristics in 
the state administrative data files that were used to construct the sampling frame. 
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Table C3. Number and percentage of programs participating in the Iowa Quality Rating System that 
participated or did not participate in the fall 2014 survey, by fall 2013 program characteristics 


All randomly 
selected programs 
(n = 600) 

Responding 
programs 
(n = 390) 

Nonresponding 
programs 
(n = 210) 

Percentage point 
difference between 

Relative difference 
between 
responding 


Characteristic 

Number 

Percent 

Number 

Percent Number 

Percent 

responding 
programs and all 
selected programs 

programs and all 
selected programs 3 
(percent) 

Chi-square 

p-value 

Program type 

Center 

286 

47.7 

182 

46.7 

104 

49.5 

-1.0 

-2.1 


Home 

314 

52.3 

208 

53.3 

106 

50.5 

1.0 

1.9 

.807 

Total 

600 


390 


210 





Iowa Quality Rating System rating as of fall 2013 

1 

21 

3.5 

10 

2.6 

11 

5.2 

-0.9 

-26.7 


2 

207 

34.5 

123 

31.5 

84 

40.0 

-3.0 

-8.6 


3 

116 

19.3 

81 

20.8 

35 

16.7 

1.4 

7.4 

.701 

4 

219 

36.5 

147 

37.7 

72 

34.3 

1.2 

3.3 

5 

37 

6.2 

29 

7.4 

8 

3.8 

1.3 

20.6 


Total 

600 


390 


210 





Geographic region 

Region 1 

126 

21.0 

98 

25.1 

28 

13.4 

4.1 

19.5 


Region 2 

131 

21.9 

89 

22.8 

42 

20.1 

1.0 

4.3 


Region 3 

100 

16.7 

54 

13.8 

46 

22.0 

-2.8 

-17.1 

.498 

Region 4 

128 

21.4 

78 

20.0 

50 

23.9 

-1.4 

-6.4 

Region 5 

114 

19.0 

71 

18.2 

43 

20.6 

-0.8 

-4.3 


Total 

599 


390 


209 





Program auspice 

School-based 

48 

8.0 

19 

4.9 

29 

13.8 

-3.1 

-39.1 


Other 

552 

92.0 

371 

95.1 

181 

86.2 

3.1 

3.4 

.074 

Total 

600 


390 


210 





Bonus amount based on rating b 

No bonus 

24 

4.0 

13 

3.4 

31 

14.8 

-0.7 

-16.7 


200-400 

245 

41.0 

156 

40.2 

94 

45.0 

-0.8 

-2.0 


600-1,000 

162 

27.1 

111 

28.6 

61 

29.2 

1.5 

5.4 

.966 

1,200-2,000 

105 

17.6 

70 

18.0 

23 

11.0 

0.5 

2.6 

2,400-4,000 

61 

10.2 

38 

9.8 

0 

0.0 

-0.4 

-4.1 


Total 

597 


388 


209 





Note: Percentages may not sum to 100 because of rounding. 






a. The percentage point difference between all randomly selected programs and responding programs, divided by the percentage of all 
randomly selected programs, multiplied by 100. 

b. Bonus amounts are determined by program type, the Iowa Quality Rating System (QRS) rating, and the program's prior rating. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership 
with regional Child Care Resource and Referral agencies. 


C-3 








Table C4. Number and percentage of programs participating in the Iowa Quality Rating System that 
participated or did not participate in the fall 2014 survey, by initial program ratings 


Program earned 
at least one third 
of possible points 
on Iowa Quality 
Rating System 
scoring domain 3 

All randomly 
selected 
programs 
(n = 600) 

Number Percent 

Responding 
programs 
(n = 390) 

Number Percent 

Nonresponding 
programs 
(n = 210) 

Number Percent 

Percentage point 
difference between 
responding 
programs and all 
selected programs 

Relative difference 
between 
responding 
programs and all 
selected programs' 1 
(percent) 

Chi-square 

p-value 

Health and safety domain 

Yes 

78 

21.3 

54 

21.2 

24 

21.6 

-0.1 

-0.6 


No 

288 

78.7 

201 

78.8 

87 

78.4 

0.1 

0.2 

1.00 

Total 

366 


255 


111 





Professional development domain 

Yes 

117 

31.9 

72 

28.1 

45 

40.5 

-3.8 

-11.8 


No 

250 

68.1 

184 

71.9 

66 

59.5 

3.8 

5.5 

.359 

Total 

367 


256 


111 





Environment domain 

Yes 

61 

16.6 

41 

16.0 

20 

18.0 

-0.6 

-3.6 


No 

306 

83.4 

215 

84.0 

91 

82.0 

0.6 

0.7 

.920 

Total 

367 


256 


111 





Family and community partnerships domain 

Yes 

229 

62.4 

165 

64.5 

64 

57.7 

2.1 

3.3 


No 

138 

37.6 

91 

35.5 

47 

42.3 

-2.1 

-5.5 

.663 

Total 

367 


256 


111 






Note: Initial program ratings include Quality Rating Systems ratings from 2011, 2012, and 2013. 

a. Only programs seeking an Iowa Quality Rating System rating of 3 or higher have data on domain score points earned; so the sample 
size is lower than the full sample size. After examining the distribution of points on each domain, the study team determined that 
earning one-third or more points was a good outpoint for differentiating programs. For the health and safety domain, homes and centers 
earning seven or more points were flagged as earning at least a third of the points. For the professional development domain, homes 
earning 12 or more points and centers earning 11 or more points were flagged. For the environment domain, homes earning 8 or more 
points, and centers earning 10 or more points were flagged. For the family and community partnerships domain, homes and centers 
earning 3 or more points were flagged. Information about total points available in each domain is included in appendix B. 

b. The percentage point difference between all randomly selected programs and responding programs, divided by the percentage for all 
randomly selected programs, multiplied by 100. 

Source: Authors’ analysis of data from a survey administered in fall 2014 by the Iowa Department of Fluman Services, in partnership 
with regional Child Care Resource and Referral agencies. 


Tables C3-C4 show the percentage distribution of some defining program characteristics 
of responding and nonresponding programs. These are characteristics that may be related 
to the level of quality improvement supports measured by the survey, and, if the sample dif- 
fered significantly from the population on many of these characteristics, the results could 
be biased. The sample of programs from which survey data were obtained is similar to all 
programs in the randomly selected sample in terms of program type, Iowa QRS rating, 
region of the state, school-based program location, Iowa QRS rating bonus amount, and 
points on specific Iowa QRS domains, with no statistically significant differences in any 
of these characteristics. The differences in raw percentages are 4.1 percent or lower for all 
characteristics in the bias analyses, although the relative differences are large for low-in- 
cidence characteristics such as school-based programs (39 percent relative difference) and 
Iowa QRS ratings of 1 (27 percent relative difference). Despite larger relative differenc- 
es, the comparison of survey respondents with the full sample of randomly selected pro- 
grams on these diverse program characteristics does not show any large differences in raw 
percentages. 
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Statewide administrative data for Iowa QRS include an indicator of whether programs in 
the Iowa QRS are located in a school, but do not include information on program funding 
sources. Information about program funding sources was collected in the quality improve' 
ment survey. According to the survey data, participating programs received funding from a 
variety of sources, both public and private (table C5). Approximately one program in four 
(85 of 375 programs, or 23 percent) receives any of four types of public preschool funding: 
Statewide Voluntary Preschool, Shared Visions, Head Start, or other funding for preschool 
through the Iowa Department of Education. 

Data sources, instruments, and collection methods 

The Iowa DHS provided the data used for the study, including quality improvement survey 
data collected by state agency staff in Iowa in fall 2014 and administrative data collected 
by the Iowa DHS on programs participating in the Iowa QRS. 

The quality improvement survey. The study team and the Iowa Quality Rating System 
Oversight Committee codeveloped the quality improvement survey instrument in 2014 
(appendix D). In April 2014 they developed a matrix of constructs to include in the survey 
and created a draft survey that was reviewed by the Iowa Quality Rating System Over- 
sight Committee and the Institute of Education Sciences. The study team also conducted 
cognitive interviews with nine early childhood education program directors participating 
in the Iowa QRS, and the Iowa DHS piloted the survey with 21 early childhood educa- 
tion program directors representing diverse program types, geographical areas, and each 
Iowa QRS rating. REL Midwest worked with the Iowa Quality Rating System Oversight 
Committee to finalize the quality improvement survey on the basis of these sources of 


Table C5. Funding sources among programs participating in the Iowa Quality 

Rating System that responded to the fall 2014 survey 


Survey respondents 
(n = 375) 

Funding source 

Frequency 

Percent 

Tuition paid by families 

277 

73.87 

Copayments or fees paid by families 

168 

44.80 

Early Head Start or Head Start 

9 

2.40 

Statewide Voluntary Preschool program 

64 

17.07 

Shared Visions preschool funding 

3 

0.80 

Other funding for preschool through the Iowa Department of Education, such as 

Title 1 

17 

4.53 

Child Care Assistance (children receiving subsidies attend your program) 

193 

51.47 

Early childhood education Iowa funding 

41 

10.93 

Area Education Agency early childhood education funding 

9 

2.40 

Federal food program (Child and Adult Care Food Program) 

255 

68.00 

Early Childhood Special Education (including Individuals with Disabilities Act 

Parts B and C) 

11 

2.93 

Other Iowa Department of Human Services funding 

34 

9.07 

Note: Programs may receive funding from multiple sources among the sources listed. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 
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feedback. The quality improvement survey collected information about quality improve- 
ment activities and supports in seven categories: 

• Training. 

• Coaching. 

• Professional development topics. 

• Continuing education. 

• New staff orientation. 

• Financial incentives. 

• Nonfmancial incentives. 

The study team developed definitions for each of these categories of quality improvement 
activities and supports (table C6). In addition, the survey collected information about bar- 
riers to quality improvement, quality improvement planning and decisionmaking, program 
funding sources, and respondent characteristics. 

The survey asked about these activities in the 12-month period prior to survey adminis- 
tration. The format and response options of the survey questions included counts (such as 
number of staff who participated in specific activities), checklists (such as yes/no endorse- 
ments of supports received), extent scales (such as satisfaction ratings and ratings of the 
extent of specific barriers), and categorical responses. There were some open-ended ques- 
tions as well, but they were not analyzed for this project. 


Table C6. Definitions of quality improvement activities and supports in the fall 
2014 survey of programs participating in the Iowa Quality Rating System 


Type of quality improvement 
support or activity 

Definition 

Training 

Professional development instruction (or workshops) provided to a group 
of early childhood education program staff on specific job-related topics, 
conducted in person or in a web-based format 

Coaching 

Individualized professional development support (often called technical 
assistance or consultation) provided by an individual to early childhood 
education program staff, conducted in person, using video or web-based 
platforms, or through informal communication such as telephone, email, and 
text messaging; coaching may include classroom-level support for teaching 
staff and program-level support for management staff 

Topics of professional 
development 

Professional development is a blanket term for training and coaching; the 
survey asks about professional development topics, referring to content 
covered in training or coaching received by program staff 

Continuing education 

Credit-bearing coursework toward any credential, degree, or certificate by 
actively employed early childhood education program staff 

New staff orientation and 

mentoring 

The provision of information, training, and other supports to new staff. 

This includes mentoring, or guidance and support offered to new staff by 
colleagues or a professional mentor 

Financial incentives 

Monetary incentives that are available to programs (such as grants, awards, 
and tiered reimbursements) and to individual staff members (such as 
scholarships) 

Nonfinancial incentives 

Nonmonetary resources to support quality improvement efforts, including 
materials (such as books or games) and equipment (such as playground 
structures or cribs) that programs may receive for free or at a discount 

Source: Regional Educational Laboratory Midwest and Iowa Quality Rating System co-developed survey. 
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The section of the survey on topics of professional development collected information 
about eight topics covered in training or coaching received by program staff: 

• Health and safety practices (such as infant and child first aid and cardiopulmonary 
resuscitation, child abuse reporting, disease control, nutrition and food prepara- 
tion, playground and equipment safety). 

• Child development (such as general child development, stages of learning). 

• Classroom practices (such as instructional techniques and activities, specific cun 
ricula or technology tools, interactions with children, classroom management, 
assessment of children). 

• Program self-assessment (such as the Environment Rating Scales, continuous 
quality improvement, strengths and areas of improvement). 

• Program management (such as business practices, working with adults and staff, 
early learning program standards, training on Head Start standards, National 
Association for the Education of Young Children standards, and preK standards). 

• Family engagement (such as cultural competence, family involvement in early 
learning, encouragement of parents to attend events and parent-teacher meetings). 

• QRS participation (such as preparing for Iowa QRS participation, working toward 
a higher Iowa QRS score). 

Three versions of the survey were created: one for centers, one for homes with assistants 
or other staff in addition to the licensed providers, and one for homes with no additional 
staff. The primary differences are in terminology and in the staff categories included in 
questions and response options (see box Cl for definitions of the staff types included in the 
versions of the survey for centers and homes). The primary terminology difference is the 
reference to program in the center version and to provider in both of the home versions. 
An example of a difference in response options occurs in questions about training topics. 
For centers, the questions include response spaces for center managers, teachers, and assis- 
tants. These questions have response spaces for providers and assistants in homes with 
staff and a response space for the provider only in homes without staff. All questions on 
the survey appear in all three versions of the survey, with the exception of the following: 

• Questions about mentoring appear only in the version of the survey for centers 
on the basis of feedback from cognitive interviews and the Iowa Quality Rating 
System Oversight Committee indicating that mentoring is not offered in homes. 

• Questions about new staff orientation do not appear in the version of the survey 
for homes without assistants because they do not have staff to orient; this decision 
was based on feedback from the Iowa Quality Rating System Oversight Commit- 
tee to avoid irrelevant questions for these programs. 

Appendix D includes the final quality improvement survey that Iowa administered in fall 
2014, formatted as a table with the complete set of items and response options for the center 
version of the survey and for the versions of the survey for homes with and without assistants. 

The Iowa DHS collected survey data in partnership with the five regional Iowa CCRR 
agencies, with guidance from the study team. The Iowa DHS identified the CCRR agency 
region of each of the 600 programs that were selected randomly for survey participation 
and provided the five CCRR agency regional directors with a list of programs located 
within their region. Because all programs in the Iowa QRS had an email address listed 
in the administrative data files, the state decided to collect the data through a web survey 
only. Iowa CCRR agency staff in each of the five regions reviewed and tested all program 
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Box Cl. Definitions of staff types in homes and centers 


Definitions of staff types in centers 

Manager. Staff with a management role in the center, including center directors, assistant 
directors, supervisors, coordinators, specialists, and other managers. 

Teacher. Teaching staff with primary responsibility for their classrooms, including teachers, 
lead teachers, and co-teachers. 

Assistant teacher. Teaching staff with support roles in their classrooms, including assistant 
teachers, aides, and floating staff. 

Definitions of staff types in homes 

Provider. Registered provider, owner, or person who serves as lead caregiver in the home. 
Assistant. A paid employee who helps the provider with caregiving on a regular basis. 


email addresses for programs within their region and updated the file with correct email 
addresses where necessary prior to data collection. In October 2014 the director of each 
Iowa CCRR agency sent email invitations to each program’s email address. In addition, 
each region identified the Iowa CCRR agency consultant assigned to work with each 
program in the randomly selected sample, and the consultant sent a personal email to each 
program’s contact, encouraging the program contacts to check their inboxes (as well as 
their junk mail folders) for the survey invitation. Respondents were entered into one of 50 
lotteries for a $100 gift certificate to purchase education supplies. 

There were 390 responses to the survey (a 65 percent response rate). Among the 286 centers 
selected for the survey, 182 responses were obtained (a 64 percent response rate). Among 
the 314 homes selected for the survey, there were 208 responses (a 66 percent response 
rate). The survey response rate can be attributed at least partly to the state’s commitment 
to ensure best practices in survey research. Item response rates ranged from 92 percent 
to 100 percent. The study team did not use statistical adjustments to account for missing 
data because the item response rates were high. Items from a list of nine barriers to quality 
improvement had slightly lower response rates (92-95 percent) than other items in the 
survey (94-100 percent); however, 96 percent of the respondents completed at least one 
item from the list of nine barriers. 

The quality improvement survey collected data about respondent characteristics. There 
was little racial/ethnic diversity among survey respondents; 97 percent reported race/ 
ethnicity as White (table C7). For comparison, the statewide population estimates for 2014 
indicate that the race/ethnicity of 87 percent of the population in Iowa is White (U.S. 
Census Bureau, 2015). 

Among survey respondents, 80 percent had more than five years of experience teaching 
(table C7). For homes, the survey instructions asked the registered provider to complete 
the survey, and the survey did not collect data on the respondents’ role in homes. For 
centers the survey instructions recommended that a director or manager complete the 
survey. According to survey responses, 81 percent of center respondents were managers or 
teachers with a management role in the program, and 19 percent were teachers (table C7). 
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Table C7. Characteristics of program staff who completed the fall 2014 survey of 
programs participating in the Iowa Quality Rating System 


Survey respondents 

Characteristics of staff who completed survey 

Frequency 

Percent 

Race/ethnicity 

White 

358 

97.02 

Hispanic 

7 

1.90 

Black 

2 

0.54 

More than one race/ethnicity 

2 

0.54 

Total 

369 


Years of experience teaching 

None 

35 

9.38 

5 or fewer 

39 

10.46 

6-15 

155 

41.55 

16-25 

92 

24.66 

More than 25 

52 

13.94 

Total 

373 


Role in the program (centers only) 

Management role only (center director, assistant director, supervisor, coordinator, 
specialist, other manager) 

126 

72.83 

Teaching role only (teacher, assistant teacher) 

33 

19.08 

Combined management and teaching role 

14 

8.09 

Total 

173 



Note: Role in the program was asked only of the 182 respondents in centers, because the state expected that 
the provider would always be the respondent in homes. Percentages may not sum to 100 because of rounding. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 


Centers participating in the Iowa QRS tended to be relatively large, with an average of five 
teachers, while about 70 percent of homes had only a single provider (tables C8 and C9). 

Administrative data were shared by the Iowa Department of Human Services for this 
project. The administrative data files included data on program type, contact information, 
region, Iowa QRS rating, the date of each rating, points earned in each Iowa QRS domain, 
and total points earned if applicable (for programs with a rating of 3, 4, or 5), Environment 
Rating Scale scores if applicable (for programs with a rating of 5), and other miscellaneous 


Table C8. Average number of each staff type among centers participating in the 
Iowa Quality Rating System and responding to the fall 2014 survey 


Center staff 

Number 

Percent 
of total 

Mean 

Median 

Standard 

deviation 

Minimum 

Maximum 

Directors 

173 

96 

1.83 

1 

2.52 

0 

30 

Teachers 

178 

98 

4.55 

3 

5.39 

0 

45 

Assistant teachers 

155 

86 

6.48 

3 

10.04 

0 

80 

Total staff 

181 

100 

12.86 

8 

15.92 

1 

155 


Source: Authors’ analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 
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Table C9. Categories of staff types among homes participating in the Iowa Quality 
Rating System and responding to the fall 2014 survey 


Survey respondents 


Home staff 

Frequency 

Percent 

Total staff 

205 


One provider 

146 

71 

Multiple staff 

Two providers 

7 

3 

Two providers and assistants 3 

8 

4 

One provider and assistants 3 

44 

21 


Note: Percentages do not sum to 100 because of rounding, 
a. The number of assistants ranged from zero to four. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human 
Services, in partnership with regional Child Care Resource and Referral agencies. 


data. Although the administrative data files contain the domain scores for programs with 
a rating of 3, 4, or 5, the data files do not contain the raw data about which Iowa QRS 
standards were met to achieve the score. The Iowa DHS provided the study team with 
historical administrative data files, which contain current and expired Iowa QRS ratings 
with rating dates from 2006 (when the QRS was launched) to March 2015. 

The Iowa DHS collects these data on programs participating in the Iowa QRS for the 
purpose of assigning and tracking the Iowa QRS ratings. To enroll in the Iowa QRS, pro- 
grams complete an application that includes self-reported data on whether the program 
meets each of the Iowa QRS standards, and programs must provide specific documentation 
for some of the standards, such as photocopies of staff degrees and credentials, training 
completion certificates, and health and safety review forms signed by a state assessor. To 
apply for a rating of 5, programs also must present documentation of classroom observations 
conducted by trained observers working for Iowa State University Extension, which over- 
sees all Iowa QRS classroom observations and trains and ensures the reliability of observ- 
ers. Once programs have completed the documentation for the rating for which they wish 
to apply, an Iowa QRS regional specialist employed by the regional Iowa CCRR agencies 
reviews the completed application for accuracy and completeness of required documenta- 
tion and assigns a rating using the Iowa QRS rating standards (included in appendix B). 

Iowa QRS ratings range from 1 (lowest) to 5 (highest), and rating criteria differ for centers 
and family child care homes. The Iowa QRS rating calculation approach is referred to as a 
hybrid rating approach, which means that program ratings are determined using a combi- 
nation of mandatory standards that all programs have to meet for certain ratings (referred 
to as blocks), as well as more flexible point requirements for some ratings. Programs must 
be licensed or registered to receive a rating of 1 and also must participate in the federal 
food program and meet some professional development criteria for a rating of 2. Higher 
ratings are determined by the number of points earned in meeting standards in any of 
several domains (professional development, health and safety, environment, and family 
and community partnerships, plus leadership and administration [centers only]), and, for 
a rating of 5, programs also must earn a 5 or higher on the ERS classroom observation 
instruments. Table CIO summarizes the requirements for each rating, and the complete 
rating guidelines and standards for centers and homes are included in appendix B. 
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Table CIO. Requirements for Iowa Quality Rating System ratings 

Rating 

Type of rating 
requirement 

Specific requirements 

1 

Block 

Programs must be registered or licensed or operated by an accredited school. 

2 

Block 

All requirements for a rating of 1, plus specific requirements related to the 
federal food program, self-assessment, and professional development. 

3 

Points 

Centers must earn between 17 and 26 total points across five domains, and 
homes must earn between 14 and 18 total points across four domains, with 
at least 1 point earned in each applicable domain. In addition, programs must 
meet all requirements for a rating of 2. 

4 

Points 

Centers must earn between 27 and 33 total points across five domains, and 
homes must earn between 19 and 24 total points across four domains, with 
at least 1 point earned in each applicable domain. In addition, programs must 
meet all requirements for a rating of 2. 

5 

Points and block 

Centers must earn at least 34 total points across five domains, and homes 
must earn at least 25 total points across four domains, with at least 1 point 
earned in each applicable domain. Programs also must earn a score of 5.0 or 
higher on an independent Environment Rating Scale observation and meet all 
requirements for a rating of 2. 

Note: Domains include professional development, health and safety, environment, and family and community 
partnerships for both centers and homes, plus leadership and administration for centers only. See appendix B 
for the detailed standards used to determine points earned in each domain. 

Source: Iowa Quality Rating System rating guidelines. 


Measures used for the study: rating increases, quality improvement activities, and program 
characteristics 

The study team constructed the analysis variables using the quality improvement survey 
data and the administrative data. All analysis variables were identified and defined during 
study planning. 

The study team used Iowa Quality Rating System ratings from two points in time to 
determine whether the ratings changed. The analyses for research question 6 examine 
the relationship between quality improvement supports and activities and increases in 
Iowa QRS ratings, as well as the relationship between quality improvement supports and 
activities and increases in the Iowa QRS domain subscores, if applicable (only programs 
with a rating of 3 or higher receive domain scores). For eligible programs with ratings at 
two time points, the study team created analysis variables indicating whether the programs 
increased their rating and each of their domain subscores or remained at the highest possi- 
ble rating (because programs cannot achieve a rating higher than 5). 

Programs were eligible for inclusion in the indicator for increase in Iowa QRS rating if 
they had obtained a valid rating in January 2014 or later (after the start of the period 
in which quality improvement supports and activities were measured on the survey, from 
September 2013 to September 2014) and also one previous rating that had been assigned 
after 2010. (Iowa QRS ratings were recalibrated to their current rating criteria in 2010; so 
any earlier ratings would not be comparable with the current valid ratings.) Programs were 
flagged as increasing if the second rating was higher than the first rating or if programs 
began at the highest rating (5) and remained there (although there were no instances of 
programs remaining at the highest rating in the study sample). Of the 172 programs eligi- 
ble for the indicator for rating increase, 78 (45 percent) increased to a higher rating. The 
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rating indicator variable is binary and does not identify programs that received a lower 
rating at the second time point. Decreases in rating were rare, occurring for only four pro- 
grams in the sample (2 percent); so the study team would not be able to reliably estimate 
the relationship between quality improvement activities and rating decreases. 

Programs were eligible for inclusion in the indicators for domain increase if they were 
eligible for the indicator for increase in Iowa QRS rating and had an Iowa QRS rating 
of 3 or higher at the second time point, meaning that they would have domain scores 
available. Domain score increase indicators were created for each domain measured in 
both centers and homes (health and safety, professional development, environment, and 
family and community partnerships; see appendix B for additional information about 
these domains). Programs were flagged as increasing if the second domain score was higher 
than the first score, if programs began at the highest possible score for the program type 
and remained there (although there were no instances of this in any domain in the study 
sample), or if programs did not receive a domain score in the first rating time point but 
did receive a domain score in the second rating time point (this occurred in the case of 
41 programs). Of the 128 programs eligible for indicators for domain score increase, 83 
increased in the health and safety domain (65 percent), 84 increased in the professional 
development domain (66 percent), 70 increased in the environment domain (55 percent), 
and 91 increased in the family and community partnerships domain (71 percent). 8 

The study team constructed some variables to represent quality improvement activity, 
support, and barrier variables. The quality improvement supports and activities variables 
included some variables that were exactly as measured in the survey and some variables that 
the study team constructed using survey responses to one or more survey items. In addition, 
the study team used survey data to construct two aggregated variables representing the number 
of quality improvement supports available to the program staff and the average level of barri- 
ers experienced by programs and their staff in accessing quality improvement activities. 

The staff quality improvement support variable was drawn from survey items that asked 
whether program staff had received eight separate types of support for professional deveh 
opment or higher education: release time, flexible schedules, Teacher Education and 
Compensation Helps (T.E.A.C.H.) scholarships, financial support for education or train- 
ing costs, books and materials, transportation reimbursement, a cash award or bonus for 
degree attainment, or a salary increase for degree attainment. The study team created a 
count variable by summing the number of quality improvement supports on this list that 
the program reported their staff had received. The study team examined the ordinal coef- 
ficient alpha of the group of items to assess the internal consistency of the full set of eight 
items as well as two subgroupings of items on the basis of how costly the supports would 
be to the programs. 9 The alpha of the full set of eight items was .89, suggesting a high level 
of internal consistency. The subgroupings of items had lower internal consistency (.70 and 
.77) and were therefore not included in study analyses. The quality improvement support 
variable ranged from 0 to 8 and had a mean of 1.29 and a standard deviation of 1.72. 

The barriers to quality improvement variable drew from survey items that asked respon- 
dents to rate the effect that nine factors had on their program’s level of participation in 
professional development and other quality improvement activities, with ratings of no 
effect (1), small effect (2), medium effect (3), or large effect (4). The nine factors are cost, 
distance and travel time, timing and scheduling of training, availability of professional 
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development on needed topics, classroom coverage, support from management, staff turn- 
over, staff availability to participate in professional development, and availability of schol- 
arships. The study team created a scaled variable by using the average of the numeric 
responses to the nine items. It examined the ordinal coefficient alpha of the group of items 
to assess the internal consistency of the full set of nine items as well as two subgroup- 
ings that group items into barriers related to the logistics of the available training options 
(that is, cost, distance, timing, and availability of topics) and those that are program-level 
factors (that is, classroom coverage, support by management, staff turnover, staff avail- 
ability to participate, and scholarships). The alpha of the full set of nine items was .817, 
suggesting a reasonably high level of internal consistency. The subgroupings of items had 
somewhat lower internal consistency (.738 and .816) and were therefore not included in 
study analyses. The barriers to quality improvement variable ranged from 1 to 4 and had a 
mean of 2.54 and a standard deviation of 0.62. 

Program characteristics were gathered from administrative data and survey informa- 
tion. The program characteristics used in the study analyses were identified from admin- 
istrative data and the survey data. Administrative data on program characteristics include 
data of the Iowa CCRR agency region in which the programs were located, whether the 
program was a center or a home, and whether the program was located in a school building 
or another type of site. The survey collected data on program funding sources, and the 
study team used these data to create a variable identifying programs that received child 
care subsidies and a variable identifying programs that received public preschool funding 
(including Iowa’s Statewide Voluntary Preschool program, Shared Visions preschool, or 
other preschool funding from the Iowa Department of Education) or Head Start funding. 

Analysis methods used in the study: descriptive and multiple regression analysis methods 

This section first describes the approach to answering the first four research questions. 
Among early childhood education programs participating in the Iowa QRS: 

• What are the quality improvement activities in which they participate? 

• What topics are covered in professional development? 

• What are the supports and barriers to participation in quality improvement activities? 

• What information do they use to make decisions about quality improvement activities? 

The study team used descriptive summary statistics to address these research questions by 
using analysis variables constructed from the quality improvement survey responses and 
administrative data on program ratings and characteristics. The study team tabulated the fre- 
quency of responses to binary, count, and ordinal analysis variables on the basis of the survey 
items and calculated the mean and standard deviation of continuous and extent scale original 
analysis variables on the basis of the survey items (see detailed results tables in appendix E). 

The program subgroups examined in the study are based on program characteristics 
measured in the administrative data or in the survey, including program type (centers or 
homes) and baseline Iowa QRS ratings (ratings 1-5). The study team conducted signifi- 
cance tests of subgroup differences. 

The study team used descriptive analysis methods examining the Iowa QRS ratings at two 
time points to address research question 5: 

• Do ratings rise among programs that renew their Iowa QRS participation? 
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First, the study team identified programs in the administrative data that had been re-rated 
with a new Iowa QRS rating during the period January 2014-March 2015 and that had 
at least one previous Iowa QRS rating. The study team then cross-tabulated the first and 
second ratings. The study team examined the cross-tabulations and counted the number 
of programs with decreased ratings (defined as receiving a lower rating at the second time 
point), with a rating that remained the same (defined as staying at the same rating for pro- 
grams that began with a rating of 2, 3, or 4 [the state does not allow programs to continue 
participating in Iowa QRS if they began with a rating of 1 and are not re-rated at a higher 
rating]), or with an increased rating (defined as receiving a higher rating at the second time 
point; the study team would also have included programs that began with a rating of 5 and 
were re-rated at 5 in this group, although no programs exhibited that characteristic in the 
administrative data). The study team then used the frequency of programs in each of these 
categories (decreased rating, rating remained the same, increased rating) to produce charts 
and tables. The study team also examined increases among programs at each rating level 
at the first time point. 

The study team used a multiple regression framework to address research question 6: 

• What is the relationship between quality improvement supports and activities and 
increases in QRS ratings and domain scores? 

Before running regression models, the study team created a binary variable indicating 
whether the program increased its rating or not using the analysis results from research 
question 5. The study team created similar binary variables indicating which programs 
increased domain scores across the two time points among the subset of rated programs 
with domain scores (those with a rating of 3, 4, and 5). 

Given the binary nature of the outcomes (variables indicating changes in the Iowa QRS 
ratings and domain scores are coded 1 for an increase and 0 for no increase), the study 
team used logistic regression analysis models to regress increases in Iowa QRS ratings or 
domain scores on quality improvement supports and activities, controlling for the baseline 
Iowa QRS ratings and program characteristics, and also staff quality improvement supports 
and barriers to quality improvement in some models. The study team ran separate series of 
logistic regression models with different dependent and independent variables, as shown 
in table Cll (the model numbers correspond with those in the results tables in appendix 
E). The models examine five dependent variables related to Iowa QRS rating outcomes, as 
defined in the section above on measures used in the study: 

• The Iowa QRS rating increased from the first to the second rating. 

• The health and safety domain score increased from the first to the second rating. 

• The professional development domain score increased from the first to the second 
rating. 

• The environment domain score increased from the first to the second rating. 

• The family and community partnerships domain score increased from the first to 
the second rating. 

The main independent variables in the analyses for research question 6 included 10 
quality improvement supports and activities analysis variables (table C12). Variables were 
first entered into separate models and then entered into a single model. 
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Table Cll. Dependent and independent variables in logistic regression models for programs 


participating in the Iowa Quality Rating System, by model number 







Model 

Model Model Model 

Model 

Model 

Model 

Model 

Model 

Model 

Variable 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Dependent variables: Iowa Quality Rating System (Iowa QRS) outcomes 

Iowa QRS rating increase 

✓ 

✓ 









Health and safety domain score 



✓ 

✓ 







Professional development domain score 





✓ 

✓ 





Environment domain score 







✓ 

✓ 



Family and community partnerships 

domain score 









✓ 

✓ 

Independent variables: quality improvement supports and activities 

Management topics 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Fifteen or more training hours 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Grants 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Continuing education 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Coaching monthly or more 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Self-assessment topics 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Classroom practices topics 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Nonfinancial incentives 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Training with follow-up 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Child development topics 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Independent variables: program characteristics 

Iowa QRS rating at baseline 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Center-based program 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Public preschool or Head Start funding 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Receives child care subsidies 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Region 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Independent variables: staff supports, barriers, and planning 

Quality improvement plan 


✓ 


✓ 


✓ 


✓ 


✓ 

Count of supports 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Barriers scale 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership 

with regional Child Care Resource and Referral agencies. 










All models included covariates for program characteristics, including the following: 

• Baseline Iowa QRS rating. 

• Program is a center. 

• Program receives funding from Head Start or state preschool. 

• Program receives child care subsidies. 

• Region, which is separated into four dummy variables representing four of the five 
regions. 

Some models also controlled for staff supports and barriers to quality improvement and 
quality improvement planning variables, as follows: 

• Program has a quality improvement plan. 

• Barriers to quality improvement scale variable. 

• Staff supports for quality improvement activities count variable. 
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Table C12. Key indicators used in regression analysis on the relationship between 
quality improvement indicators and rating increases among programs participating 
in the Iowa Quality Rating System 


1 Indicator 

Definition 

Management topics 

Managers or home providers received training or coaching on management. 

Fifteen or more training 

hours 

Key staff received an average of 15 or more training hours in the previous year. 

Grants 

As a financial incentive, the program received a grant or grants for Iowa Quality 
Rating System participation or rating level or for specific quality improvements. 

Continuing education 

Teachers or home providers were pursuing a degree or credential. 

Coaching monthly or more 

Staff in the program received coaching monthly or more frequently. 

Self-assessment topics 

Managers or home providers received training or coaching on self-assessment. 

Classroom practices topics 

Teachers or home providers received training or coaching on classroom practices. 

Nonfinancial incentives 

The program received free or discounted equipment or materials. 

Training with follow-up 

Staff in the program received training with follow-up. 

Child development topics 

Teachers or home providers received training or coaching on child 
development. 

Source: Authors' analysis of survey data from Iowa Department of Human Services. 


For research question 6, the study team ran separate sets of models for each dependent 
variable, following the same four steps for each set of models. First, the study team ran 
10 separate models entering each quality improvement support or activity into the model 
alone, controlling for program characteristics (the results of the separate models are not 
included in the appendix E tables to streamline the presentation of the results). Second, 
the study team ran models with all 10 quality improvement supports and activities in the 
same model, controlling for program characteristics. Third, the study team ran the model 
from the second step, adding in the supports and barriers variables as controls. Fourth, the 
study team ran separate models with interaction terms for the baseline Iowa QRS rating 
and center-based program type for all activities with statistically significant coefficients 
in the full model to determine whether the relationships with the Iowa QRS rating or 
domain score increases varied by these program characteristics (none of the interaction 
terms were statistically significant; so the results are not included in the appendix E tables). 

In the models with domain score dependent variables, the sample size is considerably 
smaller than the samples in the models with Iowa QRS rating increase dependent vari- 
ables (109 programs in the models with domain score increases as the dependent vari- 
able and 146 programs in the models with Iowa QRS rating increases as the dependent 
variable) because only programs with a rating of 3 or higher receive domain scores. As 
a sensitivity test, the study team ran the models with Iowa QRS rating increases as the 
dependent variable with the subsample of programs associated with domain scores (the 
differences in the model results were not substantive in nature; so the sensitivity analysis 
results are not included in the appendix E tables). The models with increases in the family 
and community partnerships variable did not fit the data well (on the basis of statistically 
significant Hosmer-Lemeshow chi-square statistics indicating poor goodness of fit and low 
model specificity; so those analysis results are not included in the appendix E tables). 

The regression analysis results in table E12 in appendix E are presented as odds ratios. An 
odds ratio greater than 1.0 and statistically significant means that programs completing 
this type of quality improvement activity show a greater likelihood of increasing the rating 
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relative to programs not completing this type of quality improvement activity. In contrast, 
an odds ratio less than 1.0 and statistically significant means that programs completing 
this type of quality improvement activity show less likelihood of increasing the rating rela- 
tive to programs not completing this type of quality improvement activity. One challenge 
in interpreting odds ratios is that the magnitude of odds ratios below 1.0 is not comparable 
with the magnitude of odds ratios above 1.0 because odds ratios above 1.0 gain magnitude 
as the numbers increase, whereas odds ratios below 1.0 lose magnitude as the numbers 
decrease and approach zero. This presents a particular challenge in communicating the 
meaning of the odds ratio results. The study team addressed this challenge in the main 
body of the report by inverting the odds ratios below zero (dividing 1.0 by the odds ratio 
of increasing the rating) to produce the odds ratio of not increasing the rating. (Note that 
this is the same odds ratio that would be produced by running the logistic regression model 
with a reverse coded binary variable in which 1 represents not increasing and 0 represents 
increasing.) In the main body of the report, figure 11 in the main text combines the odds 
ratios of increasing the rating for quality improvement activities that had an odds ratio 
above 1.0 in the main model (model 1 in table E12 in appendix E) with the odds ratios of 
not increasing the rating only for the quality improvement activities that had an odds ratio 
below 1.0 in the main model. 
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Appendix D. Quality improvement survey items 


This appendix provides information about the survey items in Iowa’s survey of early child- 
hood education providers. 


Table Dl. Items in Iowa’s quality improvement survey of early childhood education providers, fall 2014 

Question 

Response type 
(and response 
options, if 
applicable) 

Question 
asked of 
centers 

Types of 
staff asked 
about for 
centers 3 

Types of 

Question staff asked 
asked of about for 
homes homes 

without without 

assistants assistants 11 

Types of 

Question staff asked 
asked of about for 
homes homes 

with with 

assistants assistants 3 

1. Are you a center or child development 
home provider? 

Binary 







2. What is the name of your program? 

String 

✓ 


✓ 


✓ 


3. How many staff work for your program 



Manager 





in each of the staff categories? 



Teacher 




Provider 


Count 

✓ 

Assistant 

✓ 

Provider 

✓ 

Assistant 

4. In the last 12 months, how many of 
each staff type were working toward any 
credential or degree, such as a Child 
Development Associate (CDA) credential, 
an early childhood certificate, or an 



Manager 





associate’s, bachelor’s, or master's 



Teacher 




Provider 

degree? 

Count 

✓ 

Assistant 

✓ 

Provider 

✓ 

Assistant 

5. In the last 12 months, did you or any 
of your staff receive a Teacher Education 
and Compensation Helps (T.E.A.C.H.) 
scholarship? 

Binary 

✓ 


✓ 


✓ 


6. Which of the following funding sources 

Check all that apply 







does your center receive? 

(multiple subitems) 

✓ 


✓ 


✓ 


6A. Tuition paid by families 

Binary 

✓ 


✓ 


✓ 


6B. Co-payments or fees paid by families 

Binary 

✓ 


✓ 


✓ 


6C. Early Head Start or Head Start 

Binary 

✓ 


✓ 


✓ 


6D. Statewide Voluntary Preschool 
program 

Binary 

✓ 


✓ 


✓ 


6E. Shared Visions preschool funding 

Binary 

✓ 


✓ 


✓ 


6F. Other funding for preschool through 
the Iowa Department of Education, such 

as Title 1 

Binary 

✓ 


✓ 


✓ 


6G. Child Care Assistance (CCA) (children 
receiving subsidies attend your program) 

Binary 

✓ 


✓ 


✓ 


6H. Early Childhood Iowa (ECI) funding 

Binary 

✓ 


✓ 


✓ 


61. Area Education Agency (AEA) early 
childhood funding 

Binary 

✓ 


✓ 


✓ 


6J. Federal Food Program (Child and Adult 
Care Food Program [CACFP]) 

Binary 

✓ 


✓ 


✓ 


6K. Early Childhood Special Education 
(including Individuals with Disabilities Act 
[IDEA] Parts B and C) 

Binary 

✓ 


✓ 


✓ 


6L. Other Iowa Department of Human 
Services funding 

Binary 

✓ 


✓ 


✓ 


(continued) 
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Table Dl. Items in Iowa’s quality improvement survey of early childhood education providers, fall 2014 

(continued) 


Question 

Response type 
(and response 
options, if 
applicable) 

Question 
asked of 
centers 

Types of 
staff asked 
about for 
centers 3 

Question 
asked of 
homes 
without 
assistants 

Types of 

staff asked Question 
about for asked of 
homes homes 

without with 

assistants 11 assistants 

Types of 
staff asked 
about for 
homes 
with 

assistants 3 

6M. Other (please specify) 

Binary 

✓ 


✓ 


✓ 


7. In the last 12 months, did your 
program receive any of the following as a 
result of participating in Iowa QRS? 

Check all that apply 
(multiple subitems) 

✓ 


✓ 


✓ 


7A. Bonus payments for achieving a 
specific Iowa QRS rating 

Binary 

✓ 


✓ 


✓ 


7B. Free or discounted equipment or 
furniture (e.g., cribs, fences) 

Binary 

✓ 


✓ 


✓ 


7C. Free or discounted instructional 
materials (e.g., books, games) 

Binary 

✓ 


✓ 


✓ 


7D. Grants to help pay for specific quality 
improvements 

Binary 

✓ 


✓ 


✓ 


7E. Grants or funding opportunities 
that require Iowa QRS participation or a 
minimum rating 

Binary 

✓ 


✓ 


✓ 


7F. Other (please specify) 

Binary 

✓ 


✓ 


✓ 


8. Do you offer an orientation for new 
teaching staff? 

Binary 

✓ 




✓ 


9. Flow many hours of orientation do you 
provide to new staff? 

Count 

✓ 




✓ 


10. Do you use the new staff orientation 
videos and resources developed by Iowa 
State University Extension? 

Binary 

✓ 




✓ 


11. Do you offer any of the following 
types of mentoring for new staff? 

Check all that apply 
(multiple subitems) 

✓ 




✓ 


11A. Formal meeting with an experienced 

teacher 

Binary 

✓ 




✓ 


11B. Formal meeting with a professional 
mentor who does not work for your 
program 

Binary 

✓ 




✓ 


11C. Informal mentoring as needed by 
colleagues 

Binary 

✓ 




✓ 


11D. Other (please specify) 

Binary 

✓ 




✓ 


12. Flave you or any of your staff received 
coaching or consultation in the last 12 

months? 

Binary 

✓ 


✓ 


✓ 


13. In the last 12 months, how many 
staff received the following types of 
coaching or consulting? 

Count 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 


(continued) 
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Table Dl. Items in Iowa’s quality improvement survey of early childhood education providers, fall 2014 

(continued) 

Question 

Response type 
(and response 
options, if 
applicable) 

Question 
asked of 
centers 

Types of 
staff asked 
about for 
centers 3 

Types of Types of 

Question staff asked Question staff asked 
asked of about for asked of about for 
homes homes homes homes 

without without with with 

assistants assistants 11 assistants assistants 3 

14. On average, how often did coaches or Ordered categorical: 
consultants interact with staff? 1 = once a year or 

less 

2 = about twice a 

year 

3 = every couple of 

months 

4 = about once a 

month 

5 = about twice a 

month 

6 = about once a 

week 

7 = more than once 

a week 

✓ 


✓ 


✓ 


15. What types of coaching or 
consultation did staff in your program 
receive? 

Check all that apply 
(multiple subitems) 

✓ 


✓ 


✓ 


15A. In-person coaching or consultation 

Binary 

✓ 


✓ 


✓ 


15B. Online (e.g., video-based or Web- 
based modules of coaching) coaching or 

consultation 

Binary 

✓ 


✓ 


✓ 


15C. Informal (e.g., informal coaching 
that may include telephone calls, emails, 
or text messages between you and a 
coach). 

Binary 

✓ 


✓ 


✓ 


16. Have you or any of your staff 
attended trainings or workshops in the 

last 12 months? 

Binary 

✓ 


✓ 


✓ 


17. In the last 12 months, what was 
the total number of hours of training or 
workshops completed by staff in your 
program? 

Count 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

18. What types of training or workshops 
did staff in your program receive? 

Check all that apply 
(multiple subitems) 

✓ 


✓ 


✓ 


18A. In-person training 

Binary 

✓ 


✓ 


✓ 


18B. In-person training with some follow- 
up 

Binary 

✓ 


✓ 


✓ 


18C. Online training or webinars 

Binary 

✓ 


✓ 


✓ 


18D. Online training or webinars with 
some follow-up 

Binary 

✓ 


✓ 


✓ 


19. Which of the following topics were 
covered by professional development 
(including trainings, workshops, coaching, 
or consultation) received by any staff of 
each type? 

Check all that apply 
(multiple subitems) 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 


(continued) 
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Table Dl. Items in Iowa’s quality improvement survey of early childhood education providers, fall 2014 

(continued) 

Question 

Response type 
(and response 
options, if 
applicable) 

Question 
asked of 
centers 

Types of 
staff asked 
about for 
centers 3 

Types of Types of 

Question staff asked Question staff asked 
asked of about for asked of about for 
homes homes homes homes 

without without with with 

assistants assistants 11 assistants assistants 3 

19A. Health and safety practices (e.g., 
infant/child first aid and CPR, child abuse 
reporting, disease control, nutrition 
and food preparation, playground and 
equipment safety) 

Binary 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

19B. Child development (e.g., general 
child development or stages of learning) 

Binary 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

19C. Classroom practices (e.g., 
instructional techniques and activities) 

Binary 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

19D. Program self-assessment 

Binary 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

19E. Program management and business 
practices 

Binary 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

19F. Family engagement or cultural 
competence 

Binary 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

19G. QRS participation (e.g., preparing 
for Iowa’s Quality Rating System [Iowa 
QRS] participation) 

Binary 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

19H. Other topic (please specify) 

Binary 

✓ 

Manager 

Teacher 

Assistant 

✓ 

Provider 

✓ 

Provider 

Assistant 

20. How much do the following 
factors affect your program’s level of 

participation in professional development Check all that apply 
and other quality improvement activities? (multiple subitems) 

✓ 


✓ 


✓ 


20A. Cost 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


20B. Distance and travel time 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


20C. Staff availability to take part in 
professional development 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


20D. Classroom coverage 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 
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Table Dl. Items in Iowa’s quality improvement survey of early childhood education providers, fall 2014 

(continued) 


Question 

Response type 
(and response 
options, if 
applicable) 

Question 
asked of 
centers 

Types of 
staff asked 
about for 
centers 3 

Question 
asked of 
homes 
without 
assistants 

Types of 
staff asked 
about for 
homes 
without 
assistants 11 

Question 
asked of 
homes 
with 

assistants 

Types of 
staff asked 
about for 
homes 
with 

assistants 3 

20E. Availability of scholarships 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


20F. Staff turnover 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


20G. Support from management 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


20H. Availability of professional 
development on needed topics 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


201. Timing and scheduling of trainings or 
workshops 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


20J. Other (please describe below) 

1 (no effect), 

2 (small effect), 

3 (medium effect), 

4 (large effect) 

✓ 


✓ 


✓ 


21. In the last 12 months, have you 
or any of your staff received any of the 
following supports for professional 
development or higher education? 

Check all that apply 
(multiple subitems) 

✓ 


✓ 


✓ 


21A. Release time during regular work 

hours 

Binary 

✓ 


✓ 


✓ 


21B. Flexible schedules 

Binary 

✓ 


✓ 


✓ 


21C. Financial support for education or 
training costs 

Binary 

✓ 


✓ 


✓ 


21D. Books and materials 

Binary 

✓ 


✓ 


✓ 


21E. Transportation reimbursement 

Binary 

✓ 


✓ 


✓ 


21F. Cash awards or bonus payments to 
staff for course or degree completion 

Binary 

✓ 


✓ 


✓ 


21G. A raise or salary adjustment based 
on completion of degree or credential 

Binary 

✓ 


✓ 


✓ 


21H. Other (please specify) 

Binary 

✓ 


✓ 


✓ 


22. Which of the following sources of 
information do you use to determine your 
professional development needs? 

Check all that apply 
(multiple subitems) 

✓ 


✓ 


✓ 


22A. Information from Iowa’s Quality 
Rating System (Iowa QRS) rating process 

Binary 

✓ 


✓ 


✓ 


22B. Quality improvement plans 

Binary 

✓ 


✓ 


✓ 
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Table Dl. Items in Iowa’s quality improvement survey of early childhood education providers, fall 2014 

(continued) 

Question 

Response type 
(and response 
options, if 
applicable) 

Question 
asked of 
centers 

Types of 
staff asked 
about for 
centers 3 

Types of Types of 

Question staff asked Question staff asked 
asked of about for asked of about for 
homes homes homes homes 

without without with with 

assistants assistants 11 assistants assistants 3 

22C. Results of monitoring visits from 
funding or accreditation agencies 

Binary 

✓ 


✓ 


✓ 


22D. Recommendations from 

consultants, coaches, specialists, or 
other technical assistance providers 

Binary 

✓ 


✓ 


✓ 


22E. Classroom observation data 

Binary 

✓ 


✓ 


✓ 


22F. Child outcomes data 

Binary 

✓ 


✓ 


✓ 


22G. Staff surveys 

Binary 

✓ 


✓ 


✓ 


22H. Staff performance review results 

Binary 

✓ 


✓ 


✓ 


221. Specific staff requests or informal 

conversations with staff 

Binary 

✓ 


✓ 


✓ 


22J. Parent surveys 

Binary 

✓ 


✓ 


✓ 


22K. Parent committee input 

Binary 

✓ 


✓ 


✓ 


22L. Informal conversations with families 

Binary 

✓ 


✓ 


✓ 


22M. Availability of trainings/training 
schedule 

Binary 



✓ 


✓ 


22N. Working with a mentor to map out 
professional development 

Binary 



✓ 


✓ 


220. Other (please specify) 

Binary 

✓ 


✓ 


✓ 


23. Does your program have a written 
quality improvement plan that includes 
quality improvement goals and planned 
activities to meet those goals? 

Binary 

✓ 


✓ 


✓ 


24. Do you feel that your program’s 
current Iowa QRS rating reflects the true 
quality of the care provided? Why or why 

not? 

String 

✓ 


✓ 


✓ 


25. Do you have any suggestions for 

the state on how to increase access to 

quality improvement supports for your 
program? 

String 

✓ 


✓ 


✓ 


26. Is there anything else you would like 
to share about the Iowa QRS? 

String 

✓ 


✓ 


✓ 


27. What is your role at the center? 

Check all that apply 
(multiple subitems) 

✓ 






27A. Director 

Binary 

✓ 






27B. Assistant Director 

Binary 

✓ 






27C. Supervisor 

Binary 

✓ 






27D. Coordinator 

Binary 

✓ 






27E. Specialist 

Binary 

✓ 






27F. Other Manager 

Binary 

✓ 






27G. Teacher 

Binary 

✓ 






27H. Assistant Teacher 

Binary 

✓ 






271. Other (please specify) 

Binary 

✓ 
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Table Dl. Items in Iowa’s quality improvement survey of early childhood education providers, fall 2014 


(continued) 






Types of 


Types of 





Question 

staff asked 

Question 

staff asked 


Response type 


Types of 

asked of 

about for 

asked of 

about for 


(and response 

Question 

staff asked 

homes 

homes 

homes 

homes 


options, if 

asked of 

about for 

without 

without 

with 

with 

Question 

applicable) 

centers 

centers 3 

assistants assistants 11 assistants assistants 3 

28. How many years of teaching 

Check all that apply 







experience do you have? 

(multiple subitems) 

✓ 


✓ 


✓ 


28A. Years 

Count 

✓ 


✓ 


✓ 


28B. Months 

Count 

✓ 


✓ 


✓ 


29. What is your gender? 

Binary 

✓ 


✓ 


✓ 


30. What is your race? 

Check all that apply 








(multiple subitems) 

✓ 


✓ 


✓ 


30A. White 

Binary 

✓ 


✓ 


✓ 


30B. Black or African-American 

Binary 

✓ 


✓ 


✓ 


30C. American Indian or Alaskan Native 

Binary 

✓ 


✓ 


✓ 


30D. Asian 

Binary 

✓ 


✓ 


✓ 


30E. Native Hawaiian or other Pacific 








Islander 

Binary 

✓ 


✓ 


✓ 


30F. Some other race (please specify) 

Binary 

✓ 


✓ 


✓ 


31. Are you of Hispanic or Latino origin? 

Binary 

✓ 


✓ 


✓ 


32. Is your program a licensed center? 

Binary 

✓ 






33. What is your program’s license or 








registration ID? 

Alphanumeric 

✓ 


✓ 


✓ 



CPR is cardiopulmonary resuscitation. ID is identifier. QRS is Quality Rating System. 

a. This column indicates the categories of staff in centers for which the respondent was asked to supply information. 

b. This column indicates the categories of staff in homes without assistants for which the respondent was asked to supply information. 

c. This column indicates the categories of staff in homes with assistants for which the respondent was asked to supply information. 

Source: Survey co-developed by Regional Educational Laboratory Midwest and the Iowa Quality Rating System Oversight Committee 
(see Faria, Hawkinson, & Metzger, in press, for the complete formatted version of the survey). 
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Appendix E. Detailed results tables 


This appendix includes tables with the detailed results of the analyses presented in the 
main body of the report. 
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Table El. Responses to fall 2014 survey on topics related to training and coaching received in past 12 months, by program characteristic 









Training activities in past 12 months 






Program 

Any staff in program 
received training 

Average training hours 
for key staff 3 meets 15- 
hour benchmark 

Any staff in program received 
training with follow-up 

Any staff in program 
received in-person training 

Any staff in program received 
web-based training 

characteristic 

Frequency Percent 

n 

Frequency Percent 

n 

Frequency Percent 

n 

Frequency Percent 

n 

Frequency 

Percent 

n 

All programs 

372 

97.38 

382 

165 

47.28 

349 

78 

20.91 

373 

356 

95.44 

373 

183 

49.06 

373 

Program type 

Centers 

178 

98.89 

180 

46 

27.06 

170 

51 

28.81 

177 

171 

96.61 

177 

106 

59.89 

177 

Homes 

194 

96.04 

202 

119 

66.48 

179 

27 

13.78 

196 

185 

94.39 

196 

77 

39.29 

196 

Significant 

No ( x 2 

= 3.03, 


Yes ( x 2 

= 54.36, 


Yes ( x 2 

= 12.72, 


No ( X 2 

= 1.05, 


Yes ( x 2 

= 15.80, 


relationship? 

P = 

.082) 

382 

P = 

.000) 

349 

P = 

.000) 

373 

P = 

.304) 

373 

P = 

000) 

373 

Iowa Quality Rating System rating, September 2014 

1 

8 

88.89 

9 

4 

57.14 

7 

2 

25.00 

8 

7 

87.50 

8 

3 

37.50 

8 

2 

119 

97.54 

122 

47 

42.73 

110 

22 

18.49 

119 

115 

96.64 

119 

61 

51.26 

119 

3 

78 

98.73 

79 

33 

44.59 

74 

14 

17.72 

79 

76 

96.20 

79 

30 

37.97 

79 

4 

139 

96.53 

144 

66 

50.00 

132 

34 

24.29 

140 

132 

94.29 

140 

74 

52.86 

140 

5 

28 

100.00 

28 

15 

57.69 

26 

6 

22.22 

27 

26 

96.30 

27 

15 

55.56 

27 

Significant 

No ( x 2 

= 4.29, 


No ( X 2 

= 2.92, 


No ( x 2 

= 1.98, 


No ( X 2 

= 2.13, 


No ( X 2 

= 5.81, 


relationship? 

P = 

.368) 

382 

P = 

.571) 

349 

P = 

.739) 

373 

P = 

.711) 

373 

P = 

214) 

373 







Coaching activities in past 12 months 






Program 

Staff in program 
received coaching 

Staff receive coaching 
once a month or more 

Staff receive 
in-person coaching 

Staff receive coaching over 
email, telephone, or text 

Staff receive video- or 
web-based coaching 

characteristic 

Frequency Percent 

n 

Frequency Percent 

n 

Frequency Percent 

n 

Frequency Percent 

n 

Frequency 

Percent 

n 

All programs 

249 

64.68 

385 

57 

15.12 

377 

220 

57.29 

384 

129 

33.59 

384 

27 

7.03 

384 

Program type 

Centers 

117 

64.64 

181 

35 

19.44 

180 

110 

61.11 

180 

49 

27.22 

180 

16 

8.89 

180 

Homes 

132 

64.71 

204 

22 

11.17 

197 

110 

53.92 

204 

80 

39.22 

204 

11 

5.39 

204 

Significant 

No ( x 2 = 

0.0002, 


Yes 

= 5.02, 


No ( X 2 

= 2.02, 


Yes ( x 

= 6.17, 


No ( x 2 

= 1,79, 


relationship? 

P = 

.989) 

385 

P = 

.025) 

377 

P = 

.155) 

384 

P = 

.013) 

384 

P = 

181) 

384 

Iowa Quality Rating System rating, September 2014 

1 

4 

44.44 

9 

1 

12.50 

8 

3 

33.33 

9 

2 

22.22 

9 

0 

0.00 

9 

2 

69 

56.56 

122 

12 

9.92 

121 

60 

49.18 

122 

40 

32.79 

122 

7 

5.74 

122 

3 

53 

67.09 

79 

9 

11.39 

79 

49 

62.03 

79 

21 

26.58 

79 

8 

10.13 

79 

4 

100 

68.49 

146 

25 

17.61 

142 

88 

60.27 

146 

55 

37.67 

146 

9 

6.16 

146 

5 

23 

79.31 

29 

10 

37.04 

27 

20 

71.43 

28 

11 

39.29 

28 

3 

10.71 

28 

Significant 

No ( x 2 

= 8.98, 


Yes ( x 2 

= 14.24, 


No ( X 2 

= 8.93, 


No ( x 2 

= 3.79, 


No ( X 2 

= 2.90, 


relationship? 

P = 

.062) 

385 

P = 

.007) 

377 

P = 

.063) 

384 

P = 

.435) 

384 

P = 

.575) 

384 


Note: For each activity the table indicates the number of programs (frequency) and percentage of programs (percent) that selected the activity and the total number of programs 
that responded to the question (n); the significance test ( x 2 ) indicates whether there are statistically significant differences in use of the activity by the program characteristic in that 
section of the table, p values are rounded to three decimal places. 

a. Key staff includes managers and teachers in centers and providers in homes. 

Source: Authors’ analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral agencies. 










Table E2. Responses to fall 2014 survey topics related to continuing education, orientation, and mentoring, by program characteristic 


Program 


Continuing education activities in past 12 months 



New staff orientation activities in past 12 months 


Any staff in 
continuing education 

Teachers (in centers) or providers 
(in homes) in continuing education 

Program provided 
new staff orientation 

Program used Iowa State University 
Extension orientation materials 

characteristic 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

All programs 

140 

36.08 

388 

108 

27.84 

388 

187 

78.24 

239 

101 

53.72 

188 

Program type 

Centers 

100 

55.25 

181 

68 

37.57 

181 

168 

92.82 

181 

101 

59.76 

169 

Homes with 













assistants 

40 

19.32 

207 

40 

19.32 

207 

19 

32.76 

58 

0 

0.00 

19 

Significant 

Yes (x 2 

= 54.04, 


Yes (x 2 

= 16.00, 


Yes (x 2 

= 93.07, 


Yes (x 2 

= 24.54, 


relationship? 

P = 

.000) 

388 

P = 

.000) 

388 

P = 

.000) 

239 

P = 

.000) 

188 

Iowa Quality Rating System rating, September 2014 

1 

4 

40.00 

10 

2 

20.00 

10 

2 

33.33 

6 

0 

0.00 

2 

2 

28 

22.58 

124 

24 

19.35 

124 

41 

69.49 

59 

15 

35.71 

42 

3 

33 

41.77 

79 

28 

35.44 

79 

39 

76.47 

51 

31 

79.49 

39 

4 

56 

38.36 

146 

38 

26.03 

146 

80 

83.33 

96 

41 

51.25 

80 

5 

19 

65.52 

29 

16 

55.17 

29 

25 

92.59 

27 

14 

56.00 

25 

Significant 

Yes (x 2 

= 22.19, 


Yes (x 2 

= 18.05, 


Yes (x 2 

= 14.58, 


Yes (x 2 

= 18.46, 


relationship? 

P = 

.000) 

388 

P = 

.001) 

388 

P = 

.006) 

239 

P = 

.001) 

188 






Mentoring activities in past 12 months 







Program 


Formal mentoring 

Formal mentoring 


Informal 


Program 

provides mentoring 


by colleagues 


by a professional 


mentoring 


characteristic 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

All programs 

183 

76.76 

241 

113 

47.28 

239 

30 

12.55 

239 

148 

61.41 

239 

Program type 

Centers 

167 

92.27 

181 

106 

58.56 

181 

28 

15.47 

181 

137 

75.69 

181 

Homes with 













assistants 

18 

30.00 

60 

7 

12.07 

58 

2 

3.45 

58 

11 

18.33 

58 

Significant 

Yes (x 2 

= 97.94, 


Yes (x 2 

= 38.09, 


Yes (x 

= 5.78, 


Yes (x 2 

= 62.56, 


relationship? 

P = 

.000) 

241 

P = 

.000) 

239 

P = 

.016) 

239 

P = 

.000) 

239 

Iowa Quality Rating System rating, September 2014 

1 

3 

50.00 

6 

2 

33.33 

6 

2 

33.33 

6 

1 

16.67 

6 

2 

41 

69.49 

59 

23 

38.98 

59 

12 

20.34 

59 

32 

54.24 

59 

3 

38 

74.51 

51 

30 

58.82 

51 

12 

23.53 

51 

31 

60.78 

51 

4 

80 

81.63 

98 

46 

46.94 

98 

4 

4.08 

98 

63 

64.29 

98 

5 

23 

85.19 

27 

12 

44.44 

27 

0 

0.00 

27 

21 

77.78 

27 

Significant 

No( z 2 

= 6.68, 


No (x 2 

= 4.91, 


Yes (x 2 

= 21.65, 


Yes (x : 

2 = 9.75, 


relationship? 

P = 

.154) 

241 

P = 

.297) 

241 

P = 

.000) 

241 

P = 

.045) 

241 


Note: Questions about new staff orientation and mentoring were asked only of centers and homes with assistants, so the survey responses exclude 149 homes without any assis- 
tants. For each activity the table indicates the number of programs (frequency) and percentage of programs (percent) that selected the activity and the total number of programs 
that responded to the question (n); the significance test (x 2 ) indicates whether there are statistically significant differences in use of the activity by the program characteristic in that 
section of the table, p values are rounded to three decimal places. 

Source: Authors’ analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral agencies. 










Table E3. Responses to fall 2014 survey on topics covered in professional development for staff, by program characteristic 

Program 

Health and safety practices 

Child development 


Classroom practices 


Program self-assessment 

characteristic 

Frequency 

Percent 

n 

Frequency Percent 

n 

Frequency Percent 

n 

Frequency Percent 

n 

All programs 

346 

91.76 

376 

318 84.57 

376 

285 75.80 

376 

206 54.79 

376 

Program type 

Centers 

168 

94.92 

177 

154 87.01 

177 

156 88.14 

177 

121 68.36 

177 

Homes 

177 

88.94 

199 

164 82.41 

199 

129 64.82 

199 

85 42.71 

199 

Significant 

Yes 

= 4.41, 


No ( x 2 = 1.52, 


Yes (x 2 = 27.75, 


Yes (x 2 = 24.88, 


relationship? 

P = 

.036) 

376 

p = .218) 

376 

p = .000) 

376 

p = .000) 

376 

Iowa Quality Rating System rating, September 2014 

1 

6 

85.71 

7 

5 71.43 

7 

5 71.43 

7 

3 42.86 

7 

2 

109 

91.60 

119 

103 86.55 

119 

83 69.75 

119 

66 55.46 

119 

3 

75 

94.94 

79 

70 88.61 

79 

57 72.15 

79 

46 58.23 

79 

4 

128 

89.51 

143 

116 81.12 

143 

113 79.02 

143 

69 48.25 

143 

5 

27 

96.43 

28 

24 85.71 

28 

27 96.43 

28 

22 78.57 

28 

Significant 

No (j 2 

= 3.16, 


No (x 2 = 3.61, 


Yes (x 2 = 10.33, 


Yes (x 2 = 9.66, 


relationship? 

P = 

.531) 

376 

p = .462) 

376 

p = .035) 

376 

p = .047) 

376 

Program 

Program management 


Family engagement 


Quality Rating System participation 

Other topics 


characteristic 

Frequency 

Percent 

n 

Frequency Percent 

n 

Frequency Percent 

n 

Frequency Percent 

n 

All programs 

203 

53.99 

376 

178 47.34 

376 

246 65.43 

376 

12 3.19 

376 

Program type 

Centers 

114 

64.41 

177 

89 50.28 

177 

127 71.75 

177 

12 6.78 

177 

Homes 

89 

44.72 

199 

89 44.72 

199 

119 59.80 

199 

0 0.00 

199 

Significant 

Yes (x 2 

= 14.61, 


No (x 2 = 1.16, 


Yes ( x 2 = 5.92, 


Yes (x 2 = 13.94, 


relationship? 

P = 

.000) 

376 

p = .281) 

376 

p = .015) 

376 

p = .000) 

376 

Iowa Quality Rating System rating, September 2014 

1 

3 

42.86 

7 

3 42.86 

7 

3 42.86 

7 

0 0.00 

7 

2 

60 

50.42 

119 

52 43.70 

119 

73 61.34 

119 

1 0.84 

119 

3 

44 

55.70 

79 

30 37.97 

79 

58 73.42 

79 

1 1.27 

79 

4 

75 

52.45 

143 

73 51.05 

143 

92 64.34 

143 

9 6.29 

143 

5 

21 

75.00 

28 

20 71.43 

28 

20 71.43 

28 

1 3.57 

28 

Significant 

No (x 2 

= 6.16, 


Yes ( x 2 = 10.78, 


No (x 2 = 5.20, 


No (x 2 = 7.78, 


relationship? 

P = 

.187) 

376 

p = .029) 

376 

p = .267) 

376 

p = .100) 

376 


Note: For each topic of professional development the table indicates the number of programs (frequency) and percentage of programs (percent) that received it and the total number 
of programs that responded to the question (n); the significance test (x 2 ) indicates whether there are statistically significant differences in professional development on the topic by 
the program characteristic in that section of the table, p values are rounded to three decimal places. 

Source: Authors’ analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral agencies. 










Table E4. Responses to fall 2014 survey topics related to professional development received by staff, by program and staff type 


Professional 
development topic 



Centers 





Homes 


Managers (n 

Frequency 

= 169) 

Percent 

Teachers (n 

Frequency 

= 175) 

Percent 

Assistant teachers (n = 152) 

Frequency Percent 

Providers (n 

Frequency 

= 199) 

Percent 

Assistants (n 

Frequency 

= 58) 

Percent 

Health and safety 











practices 

144 

85.21 

154 

88.00 

127 

83.55 

177 

88.94 

22 

37.93 

Child development 

122 

72.19 

141 

80.57 

113 

74.34 

164 

82.41 

11 

18.97 

Classroom practices 

104 

61.54 

146 

83.43 

94 

61.84 

128 

64.32 

6 

10.34 

Program 











self-assessment 

102 

60.36 

68 

38.86 

38 

25.00 

85 

42.71 

2 

3.45 

Management 

105 

62.13 

38 

21.71 

14 

9.21 

84 

42.21 

3 

5.17 

Family engagement 

76 

44.97 

67 

38.29 

38 

25.00 

89 

44.72 

3 

5.17 

Iowa Quality Rating 











System participation 

111 

65.68 

66 

37.71 

30 

19.74 

119 

59.80 

3 

5.17 

Other 

8 

4.73 

9 

5.14 

6 

3.95 

0 

0.00 

0 

0.00 


Note: The table indicates the number of programs (frequency) and percentage of programs (percent) that had each staff type receive professional development on the indicated 
topic and the total number of programs that responded to the question (n). The number of programs that responded in each staff category, indicated by n, varies because some 
programs do not have all listed staff types. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral 
agencies. 


on 




Table E5. Responses to fall 2014 survey topics related to incentives received by programs in past 12 months, by program characteristic 


Program 

Program received 
any type of incentive 

Received free or discounted 
equipment or materials 

Received grants for Quality Rating 
System or quality improvement 

Received monetary award 
based on Quality Rating 

System rating 

characteristic 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency Percent 

n 

All programs 

287 

74.16 

387 

62 

16.02 

387 

149 

38.50 

387 

215 55.56 

387 

Program type 

Centers 

143 

79.01 

181 

31 

17.13 

181 

64 

35.36 

181 

115 63.54 

181 

Homes 

144 

69.90 

206 

31 

15.05 

206 

85 

41.26 

206 

100 48.54 

206 

Significant 

No (f 

= 4.17, 


No ( x 2 = 

0.31, 


No ( ; x 2 = 

1.42, 


Yes (x 2 = 8.77, 


relationship? 

P = 

.041) 

387 

p = .578) 

387 

p = .234) 

387 

p = .003) 

387 

Iowa Quality Rating System rating, September 2014 

1 

4 

44.44 

9 

3 

33.33 

9 

3 

33.33 

9 

0 0.00 

9 

2 

83 

66.94 

124 

31 

25.00 

124 

50 

40.32 

124 

61 49.19 

124 

3 

65 

82.28 

79 

12 

15.19 

79 

25 

31.65 

79 

50 63.29 

79 

4 

111 

76.03 

146 

13 

8.90 

146 

58 

39.73 

146 

82 56.16 

146 

5 

24 

82.76 

29 

3 

10.34 

29 

13 

44.83 

29 

22 75.86 

29 

Significant 

Yes (x 2 

= 11.63, 


Yes (x 2 = 

15.67, 


No (x 2 = 

: 2.43, 


Yes (x 2 = 20.06, 


relationship? 

P = 

.020) 

387 

p = .004) 

387 

p = .658) 

387 

p = .000) 

387 


Note: For each type of incentive the table indicates the number of programs (frequency) and percentage of programs (percent) that selected the incentive and the total number of 
programs that responded to the question (n); the significance test (x 2 ) indicates whether there are statistically significant differences in use of the incentive by the program charac- 
teristic in that section of the table, p values are rounded to three decimal places. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral 
agencies. 






Table E6. Response to fall 2014 survey topics related to quality improvement support received by staff, by program characteristic 

Program 

Financial support for 
education or training costs 

Release time during 
regular work hours 


Flexible 

schedules 



Books and 
materials 


characteristic 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

All programs 

106 

28.42 

373 

95 

25.47 

373 

81 

21.72 

373 

60 

16.09 

373 

Program type 

Centers 

89 

50.86 

175 

85 

48.57 

175 

73 

41.71 

175 

35 

20.00 

175 

Homes 

17 

8.59 

198 

10 

5.05 

198 

8 

4.04 

198 

25 

12.63 

198 

Significant 

Yes (x 2 

= 81.60, 


Yes (x 2 

= 92.69, 


Yes (x 2 

= 77.56, 


No(x 2 

= 3.74, 


relationship? 

P = 

.000) 

373 

P = 

.000) 

373 

P = 

.000) 

373 

P = 

.053) 

373 

Iowa Quality Rating System rating, September 2014 

1 

1 

14.29 

7 

1 

14.29 

7 

0 

0.00 

7 

1 

14.29 

7 

2 

25 

21.37 

117 

23 

19.66 

117 

23 

19.66 

117 

20 

17.09 

117 

3 

30 

37.97 

79 

19 

24.05 

79 

21 

26.58 

79 

8 

10.13 

79 

4 

41 

28.67 

143 

43 

30.07 

143 

28 

19.58 

143 

25 

17.48 

143 

5 

9 

33.33 

27 

9 

33.33 

27 

9 

33.33 

27 

6 

22.22 

27 

Significant 

No( x 2 

= 7.45, 


No(x 2 

= 5.10, 


No (x 2 

= 5.86, 


No (x 2 

= 3.14, 


relationship? 

P = 

.115) 

373 

P = 

.227) 

373 

P = 

.210) 

373 

P = 

.534) 

373 



Transportation 


Teacher Education and Compensation 

Cash awards to staff for course 

Raise based on completion 

Program 


reimbursement 


Helps (T.E.A.C.H.) scholarship 

or degree completion 

of degree or credential 

characteristic 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

All programs 

50 

13.40 

373 

33 

8.57 

385 

27 

7.24 

373 

27 

7.24 

373 

Program type 

Centers 

45 

25.71 

175 

17 

9.39 

181 

12 

6.86 

175 

25 

14.29 

175 

Homes 

5 

2.53 

198 

16 

7.84 

204 

15 

7.58 

198 

2 

1.01 

198 

Significant 

Yes (x 2 

= 43.03, 


No (x 2 

= 0.29, 


No (x 2 

= 0.07, 


Yes (x 2 

= 24.38, 


relationship? 

P = 

.000) 

373 

P = 

.588) 

373 

P = 

.789) 

373 

P = 

.000) 

373 

Iowa Quality Rating System rating, September 2014 

1 

0 

0.00 

7 

0 

0.00 

10 

0 

0.00 

7 

0 

0.00 

7 

2 

15 

12.82 

117 

4 

3.25 

123 

11 

9.40 

117 

4 

3.42 

117 

3 

9 

11.39 

79 

5 

6.33 

79 

3 

3.80 

79 

6 

7.59 

79 

4 

21 

14.69 

143 

16 

11.11 

144 

11 

7.69 

143 

14 

9.79 

143 

5 

5 

18.52 

27 

8 

27.59 

29 

2 

7.41 

27 

3 

11.11 

27 

Significant 

No (x 2 

= 2.20, 


Yes (x 2 

= 20.45, 


No (x 2 

= 2.80, 


No(x 2 

= 5.09, 


relationship? 

P = 

.698) 

373 

P = 

.000) 

373 

P = 

.592) 

373 

P = 

.278) 

373 


Note: For each type of quality improvement support the table indicates the number of programs (frequency) and percentage of programs (percent) that received it and the total num- 
ber of programs that responded to the question (n); the significance test (x 2 ) indicates whether there are statistically significant differences in quality improvement supports by the 
program characteristic in that section of the table, p values are rounded to three decimal places. 

Source: Authors’ analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral agencies. 










Table E7. Response to fall 2014 survey question on support and barrier factors that affect participation in professional development and 
other improvement activities, by program characteristic 



All supports 




All barriers 


Program 

characteristic 

Average level 
of supports 

Percent 

n 

Average level 
of barriers 

Percent 

n 

All programs 

1.29 

1.72 

370 

2.54 


0.62 

361 

Program type 

Centers 

2.18 

1.87 

175 

2.80 


0.56 

175 

Homes 

0.50 

1.08 

195 

2.29 


0.56 

186 

Significant 

Yes (F = 114.09, 




Yes (F= 73.86, 



relationship? 

p = .000) 


370 


p = .000) 


361 

Iowa Quality Rating System rating, September 2014 

1 

0.43 

0.79 

7 

2.29 


0.56 

7 

2 

1.08 

1.73 

116 

2.46 


0.65 

113 

3 

1.28 

1.48 

79 

2.50 


0.61 

79 

4 

1.41 

1.80 

141 

2.62 


0.57 

134 

5 

1.89 

1.95 

27 

2.60 


0.65 

28 

Significant 

No ( F= 1.89, 




No (F = 1.54, 



relationship? 

p = .111) 


370 


p = .190) 


361 


Note: For each type of quality improvement support the table indicates the number of programs (frequency) and percentage of programs (percent) that received it and the total num- 
ber of programs that responded to the question (n); the significance test (^ 2 ) indicates whether there are statistically significant differences in quality improvement supports by the 
program characteristic in that section of the table, p values are rounded to three decimal places. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral 
agencies. 






Table E8. Responses to fall 2014 survey on topics related to sources of information used to determine professional development needs, 
by program characteristic 

Program 

Information from 

QRS rating process 

Recommendations from 
consultants or coaches 

Quality 

improvement plan 


Parent 

surveys 


characteristic 

Frequency 

Percent 

n 

Frequency Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

All programs 

280 

75.07 

373 

195 52.28 

373 

181 

48.53 

373 

137 

36.73 

373 

Program type 

Centers 

138 

78.86 

175 

102 58.29 

175 

99 

56.57 

175 

80 

45.71 

175 

Homes 

142 

71.72 

198 

93 46.97 

198 

82 

41.41 

198 

57 

28.79 

198 

Significant 

No( x 2 

= 2.53, 


Yes ( x 2 = 4.77, 


Yes (x 2 

= 8.54, 


Yes (x 2 

= 11.45, 


relationship? 

P = 

.112) 

373 

p = .029) 

373 

p = .003) 

373 

P = 

.001) 

373 

Iowa Quality Rating System rating, September 2014 

1 

4 

57.14 

7 

4 57.14 

7 

0 

0.00 

7 

0 

0.00 

7 

2 

77 

65.81 

117 

60 51.28 

117 

49 

41.88 

117 

31 

26.50 

117 

3 

68 

86.08 

79 

39 49.37 

79 

34 

43.04 

79 

41 

51.90 

79 

4 

108 

75.52 

143 

70 48.95 

143 

77 

53.85 

143 

50 

34.97 

143 

5 

23 

85.19 

27 

22 81.48 

27 

21 

77.78 

27 

15 

55.56 

27 

Significant 

Yes (x 2 

= 13.16, 


Yes (x 2 = 10.25, 


Yes (x 2 

= 20.49, 


Yes (x 2 

= 21.47, 


relationship? 

P = 

.010) 

373 

p = .036) 

373 

p = .000) 

373 

P = 

000) 

373 


Informal conversations 

Classroom 


Results of monitoring visits from 


Child 


Program 


with families 


observation data 

funding or accreditation agencies 

outcomes data 


characteristic 

Frequency 

Percent 

n 

Frequency Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

All programs 

130 

34.85 

373 

124 33.24 

373 

82 

21.98 

373 

44 

0.12 

373 

Program type 

Centers 

45 

25.71 

175 

91 52.00 

175 

51 

29.14 

175 

30 

17.14 

175 

Homes 

85 

42.93 

198 

33 16.67 

198 

31 

15.66 

198 

14 

7.07 

198 

Significant 

Yes (x 2 

= 12.12, 


Yes (x 2 = 52.26, 


Yes (x 2 

= 9.85, 


Yes (x 2 

= 9.06, 


relationship? 

P = 

.000) 

373 

p = .000) 

373 

p = .002) 

373 

P = 

003) 

373 

Iowa Quality Rating System rating, September 2014 

1 

0 

0.00 

7 

1 14.29 

7 

2 

28.57 

7 

1 

14.29 

7 

2 

43 

36.75 

117 

33 28.21 

117 

21 

17.95 

117 

8 

6.84 

117 

3 

23 

29.11 

79 

22 27.85 

79 

15 

18.99 

79 

10 

12.66 

79 

4 

53 

37.06 

143 

52 36.36 

143 

35 

24.48 

143 

19 

13.29 

143 

5 

11 

40.74 

27 

16 59.26 

27 

9 

33.33 

27 

6 

22.22 

27 

Significant 

No ( X 2 

= 5.80, 


Yes (x 2 = 12.37, 


No(x 2 

= 4.25, 


No(x 2 

= 5.99, 


relationship? 

P = 

.215) 

373 

p = .015) 

373 

P = ■ 

374) 

373 

P = 

.200) 

373 

(continued) 










E-10 


Table E8. Responses to fall 2014 survey on topics related to sources of information used to determine professional development needs, 


by program characteristic (continued) 

Program 

Parent committee 
input 

Specific staff requests or 
informal conversations 

Staff performance 
review results 3 


Staff 

surveys 3 


characteristic 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

Frequency 

Percent 

n 

All programs 

27 

7.24 

373 

93 

39.91 

233 

85 

36.48 

233 

76 

32.62 

233 

Program type 

Centers 

23 

13.14 

175 

90 

51.43 

175 

85 

48.57 

175 

76 

43.43 

175 

Homes 

4 

2.02 

198 

3 

5.17 

58 

0 

0.00 

58 

0 

0.00 

58 

Significant 

Yes (x 2 

= 17.12, 


Yes (x 2 

= 38.86, 


Yes (x 2 

= 44.35, 


Yes (x 2 = 

37.38, 


relationship? 

P = 

.000) 

373 

P = 

000) 

233 

P = 

.000) 

233 

p = .000) 

233 

Iowa Quality Rating System rating, September 2014 

1 

0 

0.00 

7 

1 

25.00 

4 

0 

0.00 

4 

0 

0.00 

4 

2 

13 

11.11 

117 

22 

38.60 

57 

25 

43.86 

57 

17 

29.82 

57 

3 

4 

5.06 

79 

13 

25.49 

51 

22 

43.14 

51 

21 

41.18 

51 

4 

8 

5.59 

143 

46 

47.92 

96 

31 

32.29 

96 

31 

32.29 

96 

5 

2 

7.41 

27 

11 

44.00 

25 

7 

28.00 

25 

7 

28.00 

25 

Significant 

No (f 

= 4.29, 


No (x 2 

= 7.57, 


No ( x 2 

= 6.11, 


No (x 2 = 

= 4.08, 


relationship? 

P = 

.368) 

373 

P = • 

,108) 

233 

P = 

.191) 

233 

p = .395) 

233 


Note: For each source of information the table indicates the number of programs (frequency) and percentage of programs (percent) that used it and the total number of programs 
that responded to the question (n); the significance test (j 2 ) indicates whether or not there are statistically significant differences in information sources by the program characteris- 
tic in that section of the table. Due to an error in survey administration, data are not available for survey items on availability/schedule of trainings or working with a mentor, p values 
are rounded to three decimal places. 

a. These categories apply to centers and homes with assistants only. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral 
agencies. 






E-ll 


Table E9. Changes in Iowa Quality Rating System ratings from the first to the second rating time point for programs with at least two 
ratings, by first rating level, 2011-15 

Second 

rating 




First Iowa Quality Rating System rating 




n 

Started at 1 

Frequency Percent 

Started at 2 

Frequency Percent 

Started at 3 

Frequency Percent 

Started at 4 

Frequency Percent 

Started at 5 

Frequency Percent 

Second Quality Rating System rating 

1 

0 

0.00 

0 

0.00 

0 

0.00 

0 

0.00 

0 

0.00 

0 

2 

3 

75.00 

37 

43.02 

0 

0.00 

3 

5.77 

0 

0.00 

43 

3 

0 

0.00 

40 

46.51 

8 

27.59 

0 

0.00 

0 

0.00 

48 

4 

1 

25.00 

9 

10.47 

21 

72.41 

45 

86.54 

1 

100 

77 

5 

0 

0.00 

0 

0.00 

0 

0.00 

4 

7.69 

0 

0.00 

4 

Total 

4 


86 


29 


52 


1 


172 

Type of change at second rating 

Lower rating 

na 

na 

0 

0.00 

0 

0.00 

3 

5.77 

1 

100 

4 

Same rating 

0 

0.00 

37 

43.02 

8 

27.59 

45 

86.54 

0 

0.00 

90 

Increased rating 

4 

100.00 

49 

56.98 

21 

72.41 

4 

7.69 

na 

na 

78 

Total 

4 


86 


29 


52 


i 


172 


na is not applicable. 

Note: The table indicates the number of programs (frequency) and percentage of programs (percent) in each category, and the total number of programs that responded to the ques- 
tion ( n ). 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral 
agencies. 





E-12 


Table E10. Characteristics of programs that did and those that did not increase their Iowa Quality Rating System ratings and domain 
scores, 2011-15 










Iowa Quality Rating System domain scores 






Overall Iowa Quality 







Family and community 


Professional 



Rating System rating 

Health and safety score 

Environment score 

partnerships score 

development score 


Did not 

Did 


Did not 

Did 


Did not 

Did 


Did not 

Did 


Did not 

Did 


Program 

increase increase 

Total 

increase 

increase 

Total 

increase increase 

Total 

increase increase 

Total 

increase increase 

Total 

characteristic 

(percent) (percent) (number) 

(percent) 

(percent) (number) 

(percent) (percent) (number) 

(percent) (percent) (number) 

(percent) (percent) (number) 

All programs 

54.11 

45.89 

146 

33.03 

66.97 

109 

45.87 

54.13 

109 

26.61 

73.39 

109 

34.86 

65.14 

109 

Program type 

Centers 

55.22 

44.78 

67 

34.55 

65.45 

55 

40.00 

60.00 

55 

23.64 

76.36 

55 

40.00 

60.00 

55 

Homes 

53.16 

46.84 

79 

31.48 

66.97 

54 

51.85 

48.15 

54 

29.63 

70.37 

54 

29.63 

70.37 

54 

Significant 


No (x 2 = 0.06, 


No (x 2 = 0.12, 



No (x 2 = 1.54, 



No ( x 2 = 0.50, 



No (x 2 = 1.29, 


relationship? 


p = .803) 



p = .734) 



p = .214) 



p = .479) 



p = .256) 


Public preschool funding 

Public 

preschool 

funds 

43.24 

56.76 

37 

40.00 

60.00 

35 

25.71 

74.29 

35 

22.86 

73.39 

35 

31.43 

68.57 

35 

No public 
preschool 

funds 

57.80 

42.20 

109 

29.73 

70.27 

74 

55.41 

44.59 

74 

28.38 

71.62 

74 

36.49 

63.51 

74 

Significant 


No (x 2 = 2.36, 


No (x 2 = 1.13, 


Yes (x 2 = 8.44, 



No ( x 2 = 0.37, 



No (x 2 = 0.27, 


relationship? 


p = .125) 



p = .287) 



p = .004) 



p = .542) 



p = .605) 


Receipt of child care subsidies 

Receives 
















subsidies 

46.58 

53.42 

73 

43.40 

76.79 

53 

42.86 

57.14 

56 

21.43 

78.57 

56 

25.00 

75.00 

56 

Does not 
















receive 
















subsidies 

61.64 

38.36 

73 

23.21 

56.60 

56 

49.06 

50.94 

53 

32.08 

67.92 

53 

45.28 

54.72 

53 

Significant 


No (x 2 = 3.34, 


Yes ( x 2 = 5.01, 



No (x 2 = 0.42, 



No (x 2 = 1.58, 


Yes { x 2 = 4.93, 


relationship? 


p = .068) 



p = .025) 



p = .516) 



p = .209) 



p = .026) 


Iowa Quality Rating System rating at first rating time point 

1 

0.00 

100.00 

2 

na 

na 

na 

na 

na 

na 

na 

na 

na 

na 

na 

na 

2 

43.84 

56.16 

73 

0 

100.00 

41 

0.00 

100.00 

41 

0.00 

100.00 

41 

0.00 

100.00 

41 

3 

28.57 

71.43 

28 

42.86 

57.14 

28 

67.86 

32.14 

28 

39.29 

60.71 

28 

35.71 

64.29 

28 

4 

90.48 

9.52 

42 

58.97 

41.03 

39 

79.49 

20.51 

39 

43.59 

56.41 

39 

69.23 

30.77 

39 

5 

100.00 

0.00 

1 

100.00 

0 

1 

0 

100.00 

1 

100.00 

0 

1 

100.00 

0 

1 

Significant 

Yes ( x 2 = 36.03, 


Yes (x 2 = 35.34, 


Yes (x 2 = 58.79, 


Yes (x 2 = 25.69, 


Yes ( x 2 = 44.11, 


relationship? 


p = .000) 



p = .000) 



p = .000) 



p = .000) 



p = .000) 



(continued) 
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Table E10. Characteristics of programs that did and those that did not increase their Iowa Quality Rating System ratings and domain 
scores, 2011-15 (continued) 










Iowa Quality Rating System domain scores 






Overall Iowa Quality 
Rating System rating 

Health and safety score 

Environment score 

Family and community 
partnerships score 

Professional 
development score 

Program 

characteristic 

Did not 
increase 
(percent) 

Did 

increase 

(percent) 

Total 

(number) 

Did not 
increase 
(percent) 

Did 

increase 

(percent) 

Total 

(number) 

Did not 
increase 
(percent) 

Did 

increase 

(percent) 

Total 

(number) 

Did not 
increase 
(percent) 

Did 

increase 

(percent) 

Total 

(number) 

Did not 
increase 
(percent) 

Did 

increase 

(percent) 

Total 

(number) 

Region 

1 

43.48 

56.52 

46 

31.43 

68.57 

35 

28.57 

71.43 

35 

17.14 

82.86 

29 

31.43 

68.57 

35 

2 

65.79 

34.21 

38 

34.48 

65.52 

39 

65.52 

34.48 

29 

34.48 

65.52 

19 

41.38 

58.62 

29 

3 

66.67 

33.33 

18 

45.45 

54.55 

11 

45.45 

54.55 

11 

18.18 

81.82 

9 

27.27 

72.73 

11 

4 

57.14 

42.86 

21 

31.25 

68.75 

16 

50.00 

50.00 

16 

31.25 

68.75 

11 

43.75 

56.25 

16 

5 

43.48 

56.52 

23 

27.78 

72.22 

18 

44.44 

55.56 

18 

33.33 

66.75 

12 

27.78 

72.22 

18 


Significant No ( z 2 = 6.44, No (f = 1.08, No {f = 8.85, No ( f = 3.52, No ( f = 1.96, 

relationship? p = .168) p = .897) p = .065) p = .475) p = .744) 


na is not applicable. 

Note: The table indicates the percentage of programs that did and did not increase their Iowa QRS ratings or domain scores among programs with each characteristic and the total 
number of programs in the sample with each characteristic (n); the significance test (^ 2 ) indicates whether or not there are statistically significant differences in increasing the Iowa 
QRS rating or domain score by the program characteristic in that section of the table, p values are rounded to three decimal places. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral 
agencies. 
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Table Ell. Quality improvement supports and activities reported on the fall 2014 survey by programs that did and those that did not 
increase their ratings and domain scores, 2011-15 






Iowa Quality Rating System domain scores 





Overall Iowa Quality 
Rating System rating 

Health and safety score 

Environment score 

Family and community 
partnerships score 

Professional 
development score 


Did not Did 

Did not Did 


Did not Did 


Did not Did 


Did not Did 


Program 

characteristic 

increase increase Total 

(percent) (percent) (number) 

increase increase Total 

(percent) (percent) (number) 

increase increase Total 

(percent) (percent) (number) 

increase increase Total 

(percent) (percent) (number) 

increase increase Total 

(percent) (percent) (number) 

Fifteen or more training hours 

Yes 

52.86 47.14 70 

35.71 64.29 

56 

51.79 48.21 

56 

23.21 76.79 

56 

33.93 66.07 

56 

No 

55.26 44.74 76 

30.19 69.81 

53 

39.62 60.38 

53 

30.19 69.81 

53 

35.85 64.15 

53 

Significant 

No ( x 2 =0.08, 

No (x 2 = 0.376, 


No (x 2 = 1.62, 


No (x 2 = 0.68, 


No (x 2 = 0.04, 


relationship? 

P = -771) 

p= .540) 


p= .203) 


p= .410) 


p= .833) 


Training with follow-up 

Yes 

70.59 29.41 34 

41.67 66.97 

24 

45.83 54.17 

24 

25.00 75.00 

24 

54.17 45.83 

24 

No 

49.11 50.89 112 

30.59 69.41 

85 

45.88 54.12 

85 

27.05 72.94 

85 

29.41 70.59 

85 

Significant 

Yes ( x 2 = 4.85, 

No (x 2 = 1.04, 


No (x 2 = 0.00, 


No (x 2 = 0.04, 


Yes (x 2 = 5.05, 


relationship? 

p = .028) 

p= .308) 


p= .997) 


p= .840) 


p= .025) 


Coaching monthly or more 

Yes 

63.16 45.89 19 

52.94 47.06 

17 

58.82 41.18 

17 

29.41 70.59 

17 

58.82 41.18 

17 

No 

52.76 47.24 127 

29.35 70.65 

92 

43.48 56.52 

92 

26.09 73.91 

92 

30.43 69.57 

92 

Significant 

No (x 2 = 0.72, 

No (x 2 = 3.61, 


No (x 2 = 1.36, 


No (x 2 = 0.08, 


Yes (x 2 = 5.09, 


relationship? 

p = .396) 

p= .057) 


P= -243) 


P= -776) 


p= .024) 


Child development professional development 

Yes 

56.15 43.85 130 

33.68 66.32 

95 

49.47 50.53 

95 

27.37 72.63 

95 

34.74 65.26 

95 

No 

37.50 62.50 16 

28.57 71.43 

14 

21.43 78.57 

14 

21.43 78.57 

14 

35.71 64.29 

14 

Significant 

No (x 2 = 2.00, 

No (x 2 = 0.14, 


Yes (x 2 = 3.87, 


No (x 2 = 0.22, 


No (x 2 = 0.01, 


relationship? 

p = .158) 

p= .704) 


p= .049) 


p= .639) 


p= .943) 


Classroom practices professional development 

Yes 

52.68 47.32 112 

32.95 67.05 

88 

45.45 54.55 

88 

27.27 72.73 

88 

36.36 63.64 

88 

No 

58.82 41.18 34 

33.33 66.67 

21 

47.62 52.38 

21 

23.81 76.19 

21 

28.57 71.43 

21 

Significant 

No (x 2 = 0.40, 

No (x 2 = 0.00, 


No (x 2 = 0.03, 


No (x 2 = 0.10, 


No (x 2 = 0.45, 


relationship? 

p = .529) 

p= .974) 


p= .858) 


P= -747) 


p= .501) 


Self-assessment professional development 

Yes 

45.95 54.05 74 

25.00 75.00 

56 

39.29 60.71 

56 

16.07 83.93 

56 

30.36 69.64 

56 

No 

62.50 37.50 72 

41.51 58.49 

53 

52.83 47.17 

53 

37.74 62.26 

53 

39.62 60.38 

53 

Significant 

Yes (x 2 = 4.03, 

No (x 2 = 3.36, 


No (x 2 = 2.01, 


Yes (x 2 = 6.55, 


No (x 2 = 1.03, 


relationship? 

p = .045) 

p= .067) 


p= .156) 


p= .011) 


p= .310) 


(continued) 
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Table Ell. Quality improvement supports and activities reported on the fall 2014 survey by programs that did and those that did not 
increase their ratings and domain scores, 2011-15 (continued) 


Iowa Quality Rating System domain scores 

Overall Iowa Quality Family and community Professional 

Rating System rating Health and safety score Environment score partnerships score development score 

Did not Did Did not Did Did not Did Did not Did Did not Did 


Program increase increase Total increase increase Total increase increase Total increase increase Total increase increase Total 

characteristic (percent) (percent) (number) (percent) (percent) (number) (percent) (percent) (number) (percent) (percent) (number) (percent) (percent) (number) 


Management professional development 

Yes 

46.58 

53.42 

73 

27.12 

72.88 

59 

44.07 

55.93 

59 

25.42 

74.58 

59 

28.81 

71.19 

59 

No 

61.64 

38.36 

73 

40.00 

60.00 

50 

48.00 

52.00 

50 

28.00 

72.00 

50 

42.00 

58.00 

50 

Significant 

relationship? 


No (f = 3.34, 

p = .068) 



No (f = 2.03, 
p= .154) 



No (f = 0.17, 
p= .681) 



No (x 2 = 0.09, 
p= .726) 



No (x 2 = 2.07, 
p= .150) 


Continuing education 

Yes 

54.11 

45.89 

39 

26.47 

73.53 

34 

44.12 

55.88 

34 

17.65 

82.35 

34 

26.47 

73.53 

34 

No 

59.81 

40.19 

107 

36.00 

64.00 

75 

46.67 

53.33 

75 

30.67 

69.33 

75 

38.67 

61.33 

75 

Significant 

relationship? 


Yes (f = 5.25, 

p = .022) 



No (f = 0.96, 
P= -327) 



No (f = 0.06, 
p= .805) 



No (x 2 = 2.03, 
p= .154) 



No (x 2 = 1.53, 

p= .216) 


Nonfinancial incentives 

Yes 

64.00 

36.00 

25 

23.08 

76.92 

13 

53.85 

46.15 

13 

23.08 

76.92 

13 

38.46 

61.54 

13 

No 

52.07 

47.96 

121 

34.38 

65.63 

96 

44.79 

55.21 

96 

27.08 

72.92 

96 

34.38 

65.63 

96 

Significant 

relationship? 


No (x 2 = 1.19, 
p = .276) 



No (f = 0.66, 
p= .416) 



No (f = 0.38, 
p= .539) 



No (x 2 = 0.09, 
p= .759) 



No (x 2 = 0.08, 
P= -772) 


Grants 

Yes 

48.98 

51.02 

49 

23.68 

76.32 

38 

52.63 

47.37 

38 

21.05 

78.95 

38 

28.95 

71.05 

38 

No 

56.70 

43.30 

97 

38.03 

61.97 

71 

42.25 

57.75 

71 

29.58 

70.42 

71 

38.03 

61.97 

71 

Significant 

relationship? 


No (f = 0.78, 

P = - 377) 



No (f = 2.30, 
p= .129) 



No (x 2 = 1.07, 
p= .300) 



No (x 2 = 0.92, 
p= .337) 



No (x 2 = 0.90, 
p= .343) 



Note: The table indicates the percentage of programs that did and did not increase their Iowa QRS ratings or domain scores among programs with each characteristic and the total 
number of programs in the sample with each characteristic (n); the significance test (x 2 ) indicates whether or not there are statistically significant differences in increasing the Iowa 
QRS rating or domain score by the program characteristic in that section of the table, p values are rounded to three decimal places. 

Source: Authors' analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral 
agencies. 
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Table E12. Results of logistic regressions of increase in ratings or domain scores on program quality improvement supports and 
activities, 2011-15 


Variable 

Odds ratios for Iowa 



Odds ratios for domain score increases 



Quality Rating System 
rating increase 

Model 1 Model 2 

Health and safety 

Model 3 Model 4 

Environment 

Model 5 Model 6 

Family and community 
partnerships 

Model 7 Model 8 

Professional development 

Model 9 Model 10 

Independent variables: Quality improvement supports and activities 

Management topics 

3.01* 

4.47* 

3.41 

4.68 

3.18 

1.27 

0.86 

1.04 

3.05 

3.28 

Fifteen or more training hours 

2.90* 

2.80 

1.29 

1.67 

0.57 

0.44 

6.00* 

6.52* 

5.83* 

3.98 

Grants 

2.90 

2.62 

2.43 

2.47 

0.50 

0.38 

1.40 

1.11 

1.53 

1.21 

Continuing education 

2.24 

2.10 

0.55 

0.25 

0.31 

0.40 

1.96 

3.46 

2.32 

0.52 

Coaching monthly or more 

1.49 

1.01 

0.58 

0.43 

0.62 

0.19 

0.79 

0.59 

0.31 

0.04* 

Self-assessment topics 

1.44 

1.21 

1.36 

0.84 

2.11 

4.05 

3.55 

3.03 

0.99 

0.65 

Classroom practices topics 

0.99 

0.58 

1.56 

2.63 

3.01 

2.71 

0.55 

0.82 

0.29 

0.20 

Nonfinancial incentives 

0.82 

0.78 

0.92 

0.68 

0.41 

0.43 

1.69 

1.70 

1.18 

2.27 

Training with follow-up 

0.63 

0.57 

1.01 

0.82 

8.29 

43.06* 

1.77 

2.44 

0.55 

0.83 

Child development topics 

0.12** 

0.10** 

0.38 

0.33 

0.06* 

0.04* 

0.26 

0.25 

0.75 

1.22 

Independent variables: Program characteristics 

Quality Rating System 











rating at baseline 

q 23*** 

0.15*** 

0.11*** 

0.10*** 

0.03*** 

0.01*** 

0.16*** 

0.18** 

0.07*** 

0.03*** 

Center 

0.94 

0.89 

2.56 

4.15 

3.69 

1.33 

4.75 

7.43* 

1.04 

0.30 

Public preschool or Head 











Start funding 

3.75 

3.81 

0.34 

0.15 

8.53 

30.12* 

1.91 

1.33 

4.82 

4.87 

Receives child care 











subsidies 

1.13 

0.98 

1.27 

1.48 

0.21 

0.16 

1.23 

1.10 

3.36 

1.96 

Independent variables: Staff supports, barriers, and planning 

Quality improvement plan 


2.59 


0.70 


0.95 


0.51 


2.75 

Count of supports 


1.05 


1.19 


1.02 


0.86 


1.84* 

Barriers scale 


1.95 


0.30 


10.96* 


0.76* 


12.43* 

Model statistics 

n 

146 

137 

109 

103 

109 

103 

109 

103 

109 

103 

Likelihood ratio x 2 (model) 

59.18*** 

59.01*** 

54.16*** 

55.30*** 

92.74 

97.12*** 

41.54** 

41.06* 

69.83*** 

77.09*** 

Hosmer-Lemeshow 











X 2 (goodness of fit) 

5.70 

8.42 

9.97 

7.36 

5.61 

6.74 

8.11 

6.75 

5.67 

6.83 

Sensitivity 

74.63 

73.44 

84.93 

88.73 

89.83 

91.07 

88.75 

88.16 

91.55 

92.54 

Specificity 

83.54 

79.45 

61.11 

71.88 

88.00 

91.49 

51.72 

59.26 

78.95 

86.11 

Percentage of cases 











correctly classified 

79.45 

76.64 

77.06 

83.50 

88.99 

91.26 

78.90 

80.58 

87.16 

90.29 


* p < .05; ** p < .01; *** p < .001. 

Note: All models also control for region, which was not a significant predictor of rating increase in any model. Coefficients for each independent variable are presented as odds 
ratios. Significant odds ratios less than 1 indicate a reduced likelihood of a rating increase, and odds ratios greater than 1 indicate an increased likelihood of a rating increase. 

Source: Authors’ analysis of data from a survey administered in fall 2014 by the Iowa Department of Human Services, in partnership with regional Child Care Resource and Referral 
agencies. 







Notes 


The study team would like to acknowledge the contributions of the members of the Iowa 
Quality Rating System Oversight Committee and the Stakeholder Advisory Group in col- 
lecting the quality improvement survey data, developing the research questions for the 
study, and reviewing the study results. The study team thanks the members of the Midwest 
Early Childhood Education Research Alliance for their support of this work. In addition, 
the study team thanks colleagues Victoria Cirks, Jim Lindsay, and Jill Walston at Ameri- 
can Institutes for Research for their contributions to this report. 

1. This report includes two similar abbreviations. QRIS refers to quality rating and 
improvement systems across states, and QRS is the abbreviated name for the QRIS in 
Iowa, the Iowa Quality Rating System. 

2. Members of the Iowa Quality Rating System Oversight Committee include represen- 
tatives from the Iowa Department of Human Services, Iowa Department of Educa- 
tion, Iowa Department of Public Health, Child Care Resource and Referral agencies, 
Iowa State University Extension, Early Childhood Iowa, as well as center-, home-, and 
school-based programs. 

3. The National Institute for Early Education Research identifies a minimum of 15 hours 
of training annually for teachers as a quality benchmark (Barnett, Carolan, Squires, 
Brown, & Horowitz 2014). This study adapted the benchmark to an average of 15 
hours of training during the previous year for key staff, including managers and teach- 
ers in centers and providers in homes. Average hours of training is the sum of training 
hours among key staff, divided by the number of key staff. 

4. In registered homes, providers are required to complete 12 hours of training per year; 
assistants are not required to take a set number of hours of training. Among licensed 
centers, annual required hours of training range from 5 to 9 based on staff type. The 
state currently requires that the minimum required training hours be completed in 
person rather than online; any training beyond the required hours can be completed 
online. 

5. Bonus payments are part of QRS participation and are based on maintaining or 
improving ratings achieved by a program; programs are rated every two years, so not 
all programs would be eligible for a bonus in any given year. 

6. Quality improvement plans are an optional part of the Iowa QRS rating process. They 
confer points toward the environment domain score. Programs complete a form identi- 
fying their quality improvement goals and the specific steps they are taking to achieve 
the goals. 

7. Programs with a rating of 5 at both rating time points are not included in the category 
of rating increase because their ratings cannot possibly increase above 5. (Only one 
program in the sample started with a rating of 5, but it did not stay at that rating at the 
second time point.) Programs that start with a rating of 1 and are not able to reach a 
rating of 2 by the next rating cycle may not continue in the Iowa QRS; so no programs 
that started with a rating of 1 and remained there at the subsequent rating point are 
shown in figure 10. 

8. Domain score increases are more common in Iowa QRS than are rating increases 
because it is easier to receive additional points on a domain score than to cross to a 
higher rating. Ratings of 3-5 are determined by achieving a range of points summed 
across all domains, as well as a separate ERS observation score for a rating of 5 only 
(see appendix B for additional detail). 


Notes-1 


9. The ordinal coefficient alpha (Zumbo, Gadermann, & Zeisser, 2007) is similar to 
Cronbach’s alpha but is based on the tetrachoric correlation matrix in the case of 
binary items (such as the support variables) or on the polychoric correlation matrix 
in the case of ordinal items (such as the barrier variables). Cronbach’s alpha is not 
used with binary or ordinal items because it is based on Pearson’s r correlation matrix, 
which assumes items are normally distributed. 


Notes-2 
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The Regional Educational Laboratory Program produces 7 types of reports 


Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 



Stated Briefly 

Summaries of research findings for specific audiences 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 




